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Highlights 

Highlights  of  GAO-10-476,  a  report  to 
congressional  requesters 


ELECTIONS 

DOD  Can  Strengthen  Evaluation  of  Its  Absentee 
Voting  Assistance  Program 


Why  GAO  Did  This  Study 

Under  the  Uniformed  and  Overseas 
Citizens  Absentee  Voting  Act 
(UOCAVA),  citizens  covered  are 
permitted  to  register  and  vote 
absentee.  The  Secretary  of  Defense 
has  the  primary  responsibility  for 
federal  UOCAVA  functions,  and  the 
Department  of  Defense’s  (DOD) 
Federal  Voting  Assistance  Program 
(FVAP)  facilitates  absentee  voters’ 
participation  in  federal  elections. 
Since  2001,  the  DOD  Office  of 
Inspector  General  and  GAO  have 
reviewed  FVAP’s  efforts  and 
recommended  improvements  to  its 
procedures  and  the  direction  FVAP 
provides  to  the  services.  In 
response  to  a  congressional 
request,  this  report  evaluates  how 
FVAP  (1)  addresses  its  mission  and 
evaluates  efforts  to  conduct  it,  (2) 
aligns  budget  priorities  with 
strategic  goals,  and  (3) 
implemented  DOD  Office  of 
Inspector  General  and  GAO 
recommendations  made  from  2001 
through  2009.  GAO  analyzed 
FVAP’s  performance  measures, 
relevant  DOD  directives,  FVAP’s 
strategic  plans,  budgets,  and  past 
audit  reports.  Also,  GAO 
interviewed  agency  officials. 


What  GAO  Recommends 


GAO  recommends  that  DOD  (1) 
assess  its  current  evaluation 
methodologies,  (2)implement 
additional  methodologies,  and  (3) 
evaluate  methods  for  training 
voting  assistance  officers.  DOD 
concurred  or  partially  concurred 
with  GAO’s  recommendations. 


View  GAO-10-476  or  key  components. 

For  more  information,  contact  Jack  Edwards 
(202)  512-8246  or  edwardsj@gao.gov. 


What  GAO  Found 

FVAP  has  efforts  under  way  to  address  requirements  identified  in  DOD 
guidance,  but  FVAP’s  evaluation  of  those  efforts  yields  data  of  varying  quality. 
FVAP’s  efforts  to  address  requirements  include  obtaining  and  distributing 
state-specific  voting  information  and  developing  forms  to  request  absentee 
voting  materials.  To  assess  the  effectiveness  of  FVAP’s  efforts,  GAO  used 
criteria  (e.g.,  data  credibility,  reliability,  and  consistency)  that  it  had 
previously  identified.  While  FVAP  improved  some  of  its  evaluative 
methodologies  during  this  decade,  GAO  identified  concerns  with  findings  for 
FVAP’s  postelection  surveys  and  “measures  of  success.”  The  concerns  include 
low  response  rates  and  not  following  governmentwide  guidance  to  conduct 
nonresponse  analyses  as  well  as  credible  but  limited  measures  to  assess  some 
efforts.  Also,  FVAP  has  not  evaluated  its  Voting  Assistance  Officer  training 
even  though  Voting  Assistance  Officers  are  crucial  to  FVAP’s  voter  outreach 
efforts.  GAO  noted  the  need  for  training  programs  to  have  an  evaluative 
component.  FVAP’s  new  Director  commented  on  reasons  for  GAO  findings, 
explaining  that  many  organizations  focus  on  implementation  instead  of 
impact,  but  he  plans  to  add  more  evaluations  in  the  future.  Better  evaluative 
information  on  its  efforts  and  an  assessment  of  its  Voting  Assistance  Officer 
training  would  allow  FVAP  to  enhance  its  efforts  to  be  effective  and  efficient. 

While  a  detailed  analysis  was  not  possible  because  FVAP  does  not  budget  by 
strategic  goals,  FVAP’s  recent  budgets  and  current  strategic  goals  appear  to  be 
generally  aligned.  GAO’s  review  found  linkages  between  FVAP’s  recent  annual 
budgets  that  have  averaged  about  $4  million  and  its  strategic  goals.  The 
strategic  plan  for  2008  and  2009  contained  four  general  goals,  including 
encouraging  adoption  of  FVAP  legislative  initiatives  in  order  to  facilitate 
UOCAVA  voting  and  improving  marketing  and  outreach  efforts.  The  linkages 
to  the  goals  were  most  readily  apparent  for  the  budget  categories  of  travel  and 
contracts.  For  example,  FVAP’s  travel  budget  was  higher  during  years  with 
federal  elections,  in  large  part  due  to  travel  to  conduct  Voting  Assistance 
Officer  training  workshops  and  in  support  of  FVAP’s  objective  to  enhance 
training  products  and  services.  In  addition,  FVAP’s  contracts  budget  for 
products  and  services,  such  as  improving  the  Web  site  and  exploring  new 
technology  methods  for  Internet  voting,  are  linked  to  goals  involving  the 
improvement  of  the  FVAP’s  marketing  and  outreach  efforts  with  stakeholders. 

FVAP  and  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness  fully 
or  partially  implemented  most  UOCAVA-related  audit  recommendations  from 
2001  through  2009.  FVAP  and  the  Under  Secretary  fully  or  partially 
implemented  11  of  16  DOD  Office  of  Inspector  General  and  GAO 
recommendations  for  improvements  in  the  oversight  and  direction  it  provides 
to  the  services.  These  included  several  changes  to  DOD  guidance  to  specify 
the  number  of  Voting  Assistance  Officers  needed  and  emphasize  the  services’ 
responsibilities  in  monitoring  and  overseeing  their  programs.  Also,  FVAP  fully 
or  partially  implemented  six  of  the  seven  recommendations  for  improvements 
in  its  processes,  such  as  improving  the  security  of  its  electronic  initiatives. 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


June  17,  2010 
Congressional  Requesters 

Members  of  the  military,  their  dependents  of  voting  age,  and  other  eligible 
U.S.  citizens  living  abroad  are  permitted  to  participate  by  absentee  ballot 
in  all  federal  elections  under  the  Uniformed  and  Overseas  Citizens 
Absentee  Voting  Act  (UOCAVA).1  According  to  the  Department  of  Defense 
(DOD),  the  act  covers  over  6  million  U.S.  citizens.  The  Secretary  of 
Defense  is  the  presidential  designee  with  the  primary  responsibility  for 
federal  UOCAVA  functions;  and  through  the  Under  Secretary  of  Defense 
for  Personnel  and  Readiness  (USD  (P&R)),  the  Federal  Voting  Assistance 
Program  (FVAP)  provides  support  to  potential  overseas  absentee  voters  in 
order  to  facilitate  their  participation  in  federal  elections.  Absentee  voters 
may,  however,  experience  difficulties  accessing  voting  materials  and  then 
returning  them  in  time  to  meet  state-specific  deadlines. 

According  to  FVAP’s  2008-2009  strategic  plan,  the  program’s  mission  is  to 
inform  and  educate  U.S.  citizens  worldwide  about  their  right  to  vote; 
foster  voter  participation;  and  protect  the  integrity  of  and  enhance  the 
electoral  process  at  the  federal,  state,  and  local  levels.  Among  other  things, 
FVAP  staff  train  voting  assistance  officers  (VAOs)  from  the  services,  the 
Department  of  State,  and  overseas  citizen  organizations  to  help  these 
entities  implement  their  voting  assistance  programs.  Since  2001,  the  DOD 
Office  of  Inspector  General  and  GAO  have  reviewed  FVAP’s  efforts  and 
made  recommendations  to  USD  (P&R)  to  improve  the  oversight  and 
direction  FVAP  provides  to  the  services  and  to  improve  FVAP’s  processes. 

Following  the  2008  presidential  election,  you  requested  that  we  assess 
FVAP’s  efforts  to  provide  assistance  to  absentee  voters.  In  response  to 
your  request,  we  evaluated  the  extent  to  which  FVAP  has  (1)  addressed  its 
mission  and  evaluated  the  effectiveness  of  its  efforts  to  conduct  that 
mission,  (2)  put  processes  in  place  to  help  ensure  that  budget  priorities  are 
aligned  with  the  strategic  plans  and  goals,  and  (3)  implemented 
recommendations  made  from  2001  through  2009  by  the  DOD  Office  of 
Inspector  General  and  GAO.  This  report  contributes  to  a  larger  GAO  body 
of  work  on  DOD’s  overseas  absentee  voting  program  (see  the  Related  GAO 
Products  section  at  the  end  of  this  report). 


1  Pub.  L.  No.  99-410  (1986),  codified  as  subsequently  amended  at  42  U.S.C.  §§1973ff  et  seq. 
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To  evaluate  the  extent  to  which  FVAP  has  addressed  its  mission  and 
evaluated  the  effectiveness  of  its  efforts  to  conduct  that  mission,  we 
analyzed  FVAP’s  documentation  of  its  efforts,  such  as  communications 
with  VAOs  and  state  election  officials,  and  the  relationship  of  those  efforts 
to  FVAP’s  mission  requirements  outlined  in  UOCAVA  and  related  DOD 
guidance.  We  corroborated  our  findings  during  interviews  with  FVAP 
officials.  To  assess  the  performance  measures  that  FVAP  used  to  evaluate 
the  extent  to  which  its  efforts  addressed  its  mission,  we  reviewed  FVAP’s 
data  gathering  procedures  and  available  measures  obtained  from  FVAP 
and  published  reports  (e.g.,  ours  and  those  of  the  DOD  Office  of  Inspector 
General  and  non-governmental  organizations).  We  then  compared  FVAP’s 
procedures  and  measures  against  criteria  on  performance  measurement 
that  GAO  and  others  had  published  in  previous  reports.2  For  example,  we 
reviewed  the  methodology  and  questions  used  in  FVAP’s  2008  postelection 
surveys  of  servicemembers,  VAOs,  overseas  citizens,  and  local  election 
officials  and  compared  them  against  survey-related  criteria  in  guidance 
from  the  Office  of  Management  and  Budget.3  To  evaluate  the  extent  to 
which  FVAP  has  processes  in  place  to  align  its  budget  priorities  with  its 
strategic  plans  and  goals,  we  reviewed  FVAP’s  budget  processes  and 
compared  its  budgets  for  fiscal  years  2005  through  2009  with  the  goals 
described  in  its  strategic  plan  for  2008  through  2009.  To  determine  the 
extent  to  which  the  DOD  Office  of  Inspector  General’s  and  GAO’s 
recommendations  were  implemented,  we  reviewed  DOD’s  Office  of 
Inspector  General  and  GAO’s  reports  issued  from  2001  through  2009.  We 
also  interviewed  the  DOD  Office  of  Inspector  General  officials  who  had 
conducted  the  evaluations  of  FVAP’s  and  USD  (P&R)’s  actions  to 
implement  recommendations  and  reviewed  DOD  directives,  a  DOD  draft 
instruction,  USD  (P&R)  memorandums,  and  DOD  Office  of  Inspector 
General  follow-up  records  of  these  evaluations  to  help  determine  the 
extent  to  which  FVAP  and  USD  (P&R)  had  implemented  the  DOD  Office  of 
Inspector  General’s  recommendations.  Similarly,  we  reviewed  our  reports 
and  follow-up  records  to  determine  the  extent  to  which  GAO 
recommendations  had  been  implemented.  For  each  of  our  objectives,  we 
assessed  the  reliability  of  the  data  we  analyzed  by  reviewing  existing 
documentation  related  to  the  data  sources  and  interviewing 
knowledgeable  agency  officials  about  the  data  that  we  used.  We  found  the 


2  GAO,  Tax  Administration:  IRS  Needs  to  Further  Refine  Its  Tax  Filing  Season 
Performance  Measures,  GAO-03-143  (Washington,  D.C.:  Nov.  22,  2002). 

3  Office  of  Management  and  Budget,  Guidance  on  Agency  Survey  and  Statistical 
Information  Collection,  (Washington,  D.C.:  January  2006). 
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data  sufficiently  reliable  for  the  purposes  of  this  report.  Appendix  I 
explains  our  scope  and  methodology  in  greater  detail. 

We  conducted  this  performance  audit  from  June  2009  through  May  2010  in 
accordance  with  generally  accepted  government  auditing  standards.  Those 
standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient, 
appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence 
obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives. 


Background 


FVAP  relies  heavily  on  the  military  services  and  the  Department  of  State  to 
carry  out  mission  requirements  to  help  ensure  that  U.S.  citizens  covered 
by  UOCAVA  understand  their  voting  rights  and  how  to  register  and  vote  by 
absentee  ballot.  Each  military  service  is  also  required  to  designate  a  senior 
service  voting  representative  to  manage  the  service’s  voting  assistance 
program.  Similarly,  the  Department  of  State — through  its  Bureau  of 
Consular  Affairs,  embassies,  and  consulates — carries  out  a  portion  of  its 
voter  assistance  responsibilities  through  staff  designated  to  provide 
assistance. 

To  accomplish  its  mission,  FVAP  also  coordinates  and  interacts  with  a 
number  of  other  stakeholders  including  the  Election  Assistance 
Commission  (Commission),  state  and  local  election  officials,  and  non¬ 
governmental  organizations.  For  example,  the  Commission  was  directed  to 
establish  electronic  absentee-voting  system  guidelines  for  DOD  and  FVAP 
to  use  in  establishing  an  electronic  absentee-voting  demonstration  project 
as  directed  by  section  1604  of  the  National  Defense  Authorization  Act  for 
Fiscal  Year  2002. 4  FVAP  also  works  with  state  and  local  election  officials 


4  The  National  Defense  Authorization  Act  for  Fiscal  Year  2002,  Pub.  L.  No.  107-107,  §1604 
(2001)  directed  the  Secretary  of  Defense  to  carry  out  a  demonstration  project  under  which 
absent  uniformed  services  voters  are  permitted  to  cast  ballots  through  an  electronic  voting 
system.  Initially,  the  project  was  to  be  carried  out  in  the  general  election  for  federal  office 
in  either  November  2002  or  November  2004.  The  Ronald  W.  Reagan  National  Defense 
Authorization  Act  for  Fiscal  Year  2005,  Pub.  L.  No.  108-375,  §567  (2004)  amended  that 
requirement  by  requiring  the  project  to  be  implemented  during  the  first  general  election  for 
federal  office  that  occurs  after  the  Election  Assistance  Commission  establishes  electronic 
absentee  voting  guidelines  and  certifies  that  it  will  assist  the  Secretary  of  Defense  in 
carrying  out  the  project.  In  2007,  we  reported  that  DOD  had  not  moved  foiward  with  the 
project  because  the  Commission  had  not  developed  the  guidelines  that  DOD  needs  to 
develop  the  demonstration  project. 
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to  promote,  among  other  things,  the  adoption  of  legislative  initiatives  to 
facilitate  absentee  voting  under  UOCAVA  and  to  help  ensure  mutual 
understanding  of  state-specific  absentee-voting  procedures.  Finally,  FVAP 
coordinates  with  advocacy  groups  and  other  non-governmental 
organizations  like  the  Overseas  Vote  Foundation  and  the  Pew  Charitable 
Trusts  to  leverage  the  outreach  capacity  of  those  organizations  for 
reaching  the  diffuse  population  of  overseas  U.S.  citizens. 

Findings  from  our  past  reports  have  documented  opportunities  for 
improving  FVAP  efforts  to  meet  mission  requirements.  For  example,  in 
2001,  we  reported  on  opportunities  for  DOD  to  improve  outreach  to 
absentee  voters  and  improve  the  oversight  of  FVAP.  In  2007,  we  reported 
on  opportunities  for  DOD  to  improve  electronic  absentee  voting 
initiatives — one  opportunity’s  being  for  DOD  to  develop  action  plans  to 
identify  safeguards  for  the  security  and  privacy  of  DOD’s  electronic  and 
Internet-accessible  voting  systems. 


FVAP  Has  Taken 
Steps  to  Address  DOD 
Requirements,  but  the 
Quality  of  the 
Evaluative 
Information  Varies 


FVAP  Has  Taken  Steps  to 
Address  Requirements  in 
the  DOD  Directive  That 
Implements  UOCAVA 


FVAP  has  efforts  under  way  to  address  requirements  set  out  in  the  DOD 
guidance  implementing  the  executive  order  assigning  primary 
responsibility  for  federal  UOCAVA  functions  to  the  Secretary  of  Defense.5 
DOD  Directive  1000.04  (DOD’s  guidance)  specifies  that  USD  (P&R)  is 
responsible  for  managing,  coordinating,  and  performing  the 
responsibilities  assigned  to  the  presidential  designee  under  UOCAVA,  and 
for  administering  FVAP.6  The  directive  also  requires  USD  (P&R)  to 
designate  a  Director  for  FVAP  who  is  to  be  responsible  for  all  aspects  of 
the  program,  and  the  directive  contains  a  number  of  requirements  that 


5  42  U.S.C.  §1973ff,  etseq. 

6  DOD  Directive  1000.04,  Federal  Voting  Assistance  Program  (FVAP)  (Apr.  14,  2004). 
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address  the  responsibilities  assigned  in  UOCAVA.  Table  1  describes 
requirements  in  DOD’s  Directive  1000.04  and  identifies  FVAP’s  efforts  to 
address  them.  For  example,  to  address  the  requirement  to  publicize  the 
right  of  citizens  to  participate  in  the  electoral  process  under  UOCAVA, 
FVAP  prints  and  issues  voter  literature  (e.g.,  posters)  and  has 
downloadable  versions  of  literature  and  other  information  on  FVAP’s  Web 
site.  Among  other  things,  these  materials  provide  the  absentee  voter  with 
information  on  how  to  contact  FVAP’s  ombudsman  service,  as  well  as 
methods  and  a  timeline  for  both  requesting  an  application  for  absentee 
voting  and  returning  an  absentee  ballot. 


Table  1 :  FVAP’s  Efforts  to  Address  the  Requirements  in  DOD  Directive  1000.04 

DOD-specified  requirements 

FVAP’s  efforts  to  address  requirements 

Prescribe  an  official  postcard  form,  containing  both  an 
absentee  voter  registration  and  absentee  ballot 
registration  application  for  use  by  the  states,  as  part  of  the 
UOCAVA  responsibilities  referenced  in  the  directive. 

Developed  Federal  Post  Card  Application  (Standard  Form-76A)  that  all 
states  accept  as  an  application  for  the  absentee  voter  registration  and 
ballot. 

Prescribe  a  federal  write-in  absentee  ballot  for  use  in 
general  elections  for  federal  office  by  absent  uniformed 
services  voters  and  overseas  voters  who  make  timely 
application  for,  and  do  not  receive,  states'  absentee 
ballots  as  partofthe  UOCAVA  responsibilities  referenced 
in  the  directive. 

Developed  Federal  Write-In  Absentee  Ballot  (Standard  Form-186A)  that 
all  states  accept  for  absentee  voting  in  federal  elections. 

Prescribe  the  standard  oath  to  be  used  with  any  UOCAVA 
documents.3 

Prepared  the  standard  oath  and  prints  it  on  official  absentee  voting 
application  and  ballot. 

Designate  a  biennial  Armed  Forces  Voters  Week  to 
encourage  military  personnel  and  their  family  members  to 
exercise  their  right  to  vote. 

Identifies  and  communicates  the  dates  of  the  Armed  Forces  Voters  Week 
to  the  services. 

Obtain  current  voting  information  from  each  state,  and 
disseminate  it  to  other  federal  executive  departments, 
agencies,  and  DOD  components. 

Updates  deadlines  and  other  state-specific  requirements  in  the  Voting 
Assistance  Guide,  provides  it  to  VAOs,  and  posts  the  guide  on  the  FVAP 
Web  site. 

Publicize  the  right  of  citizens  to  participate  in  the  electoral 
process  under  UOCAVA. 

Issues  voter  literature  (e.g.,  posters),  posts  downloadable  literature  and 
other  information  on  the  FVAP  Web  site,  and  issues  press  releases  that 
publicize  information  on  UOCAVA-covered  citizens'  right  to  vote. 

Provide  an  ombudsman  service  for  all  UOCAVA-covered 
persons  and  state  and  local  officials. 

Provides  a  toll-free  telephone  ombudsman  service  to  assist  VAOs, 
absentee  voters,  and  state  and  local  officials  with  their  questions  and 
paperwork. 

Ensure  voters  receive  information  about  registration, 
voting  procedures,  and  materials  pertaining  to  upcoming 
elections. 

Distributes  Voting  Assistance  Guides,  Federal  Postcard  Applications, 
and  other  materials  to  absentee  voters  and  VAOs;  and  posts  these 
materials  on  the  FVAP  Web  site. 

Establish  and  maintain  liaison  with  officials  of  state 
legislatures  and  with  state  and  local  government  officials. 

Maintains  contact  with  state  and  local  officials,  including  issuing 
legislative  initiative  letters  and  newsletters  to  identify  how  a  state  can 
meet  requirements  and  incorporate  technology  into  the  absentee  voting 
process. 
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DOD-specified  requirements 

FVAP’s  efforts  to  address  requirements 

Encourage  and  assiststates  and  other  U.S.  jurisdictions  to 
adopt  the  mandatory  and  recommended  UOCAVA 
provisions,  and  advise  them  on  how  federal  laws  and 
regulations  apply  to  their  individual  electoral  systems. 

Provides  state-specific  legislative  initiative  letters  detailing  UOCAVA 
requirements  and  recommendations  to  accommodate  technological 
advances  in  absentee  voting. 

Conduct  voting  assistance  workshops  forVAOs  in  even- 
numbered  years  worldwide. 

Provides  in-person  and  online  training  workshops  to  train  VAOs  on  how  to 
help  potential  absentee  voters  understand  their  voting  rights. 

Survey  military  and  other  U.S.  citizens  covered  by 

UOCAVA  to  gather  information  for  the  quadrennial  report 
to  the  President  and  Congress  required  by  UOCAVA. 

Conducts  postelection  surveys  of  servicemembers,  VAOs,  overseas 
citizens,  and  local  election  officials;  and  issues  quadrennial  report  to  the 
Presidentand  Congress. 

Source:  GAO  analysis  of  DOD  data. 


“The  standard  oath  requires  voters  to  "affirm  that  a  material  misstatement  of  fact  in  the  completion  of 
such  a  document  may  constitute  grounds  for  conviction  for  perjury." 


FVAP’s  Evaluations  of  Its 
Efforts  to  Address  the 
Requirements  Yield  Data 
of  Varying  Quality 


The  evaluative  information  on  FVAP’s  efforts  varies  in  terms  of  quality.  In 
a  prior  report,  we  noted  that  agencies  need  reliable  information  on  the 
performance  of  agency  programs,  the  financial  condition  of  programs  and 
their  operations,  and  the  costs  of  programs  and  operations  in  order  to 
efficiently  and  effectively  operate,  manage,  and  oversee  activities. '  We 
also  have  identified  the  reliability,  consistency,  and  credibility  of  data  as 
some  of  the  characteristics  that  might  be  examined  in  assessing  the  quality 
of  information.8 


During  the  last  decade,  we  and  the  DOD  Office  of  Inspector  General  have 
identified  opportunities  for  FVAP  to  enhance  the  evaluative  information 
that  the  agency  has  for  monitoring  and  adjusting  its  efforts,  and  FVAP  took 
steps  to  improve  its  evaluative  information.  For  example,  in  2001,  we 
recommended  that  FVAP  develop  a  methodology  to  gather  nationally 
projectable  data  on  disqualified  military  and  overseas  absentee  ballots  and 
reasons  for  disqualification.  In  response,  FVAP  redesigned  its  postelection 
survey  of  local  election  officials  to  obtain  the  number  of  absentee  ballots 
not  counted  and  the  reasons  for  the  disqualification  of  absentee  ballots 
from  servicemembers  and  other  overseas  citizens.  Similarly,  as  part  of  its 
evaluation  of  the  services’  voting  assistance  programs  for  2007,  the  DOD 
Office  of  Inspector  General  found  that  the  services  were  not  using 
consistent  metrics  and  targets  when  reporting  on  the  effectiveness  and 


7  GAO,  Managing  for  Results:  Critical  Issues  for  Improving  Federal  Agencies’  Strategic 
Plans,  GAO/GGD-97-180  (Washington,  D.C.:  Sept.  16,  1997). 

8  GAO,  Program  Evaluation:  An  Evaluation  Culture  and  Collaborative  Partyierships  Help 
Build  Agency  Capacity,  GAO-03-454  (Washington,  D.C.:  May  2,  2003). 
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compliance  of  their  programs.  In  response  to  this  concern,  FVAP 
established  with  the  services  some  performance  metrics  that  were 
published  in  the  DOD  Office  of  Inspector  General’s  subsequent  annual 
assessment  of  the  services’  voting  assistance  programs.9  FVAP  refers  to 
these  new  performance  measures  as  its  “measures  of  success.” 

During  our  discussions  with  FVAP  officials  about  how  the  agency 
evaluates  its  efforts  to  address  the  requirements  in  the  DOD  directive,  the 
officials  most  frequently  cited  findings  from  FVAP’s  postelection  surveys 
and  measures  of  success.  As  a  result  of  our  review  of  documents  and 
discussions  with  DOD  staff  knowledgeable  about  the  methodologies,  we 
identified  concerns  regarding  some  of  the  resulting  evaluative  information. 

•  Postelection  surveys.  Following  each  presidential  election,  FVAP 
administers  variations  of  this  survey  to  six  categories  of  individuals 
covered  or  impacted  by  UOCAVA:  uniformed  servicemembers,  federal 
civilians  overseas,  non-federal  civilians  overseas,  unit  VAOs,  State 
Department  VAOs,  and  local  election  officials.  Respondents  complete  a 
survey  and  submit  the  form  directly  to  FVAP,  which  then  compiles  the 
results.  As  part  of  recent  improvements  to  its  survey  process,  FVAP 
collaborated  with  survey  professionals  in  another  part  of  DOD  to, 
among  other  things,  re-word  some  survey  questions  and  adjust 
sampling  methods  in  an  effort  to  improve  the  statistical  validity  and 
generalizability  of  the  survey  results.  Despite  these  improvements, 
concerns  remain  regarding  the  evaluative  information  resulting  from 
the  surveys.  For  example,  even  though  the  response  rates  for  some  of 
the  surveys  are  low,  FVAP  does  not  conduct  a  non-response  bias 
analysis  that  the  guidance  from  the  Office  of  Management  and  Budget 
has  identified  as  a  necessary  step  in  determining  whether  survey 
findings  are  biased.10  Not  conducting  such  an  analysis  limits  data 
reliability,  a  data  collection  characteristic  that  GAO  previously 
identified  as  affecting  information  quality.  We  also  talked  about 
credibility  of  data,  another  GAO-identified  characteristic  that  might  be 
examined  in  assessing  the  quality  of  the  survey  findings.  We  noted  that 


9  DOD  Office  of  Inspector  General,  2008  Evaluation  of  the  DOD  Voting  Assistance 
Program ,  Report  No.  IE-2009-005  (Arlington,  Va.:  Apr.  30,  2009). 

10  Office  of  Management  and  Budget’s  January  2006  Guidance  on  Agency  Survey  and 
Statistical  Information  Collections  says,  among  other  things,  that  information  collection 
requests  for  “surveys  with  expected  response  rates  lower  than  80  percent  need. .  .a 
description  of  plans  to  evaluate  nonresponse  bias.”  In  explaining  the  reason  for  such 
analyses,  the  guidance  notes  that  the  lower  the  response  rates  are  and  the  greater  the 
differences  between  respondents  and  nonrespondents,  the  greater  the  nonresponse  bias. 
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potentially  important  content  information  was  not  included  as  survey 
questions.  For  example,  FVAP’s  survey  of  election  officials  does  not 
ask  whether  the  state-specific  legislative  initiative  letters  that  FVAP 
sends  were  useful  or  how  FVAP  may  assist  states  in  better  meeting  the 
provisions  of  UOCAVA.  Without  incorporating  a  wider  selection  of 
credible,  valid  questions  into  its  surveys,  FVAP  cannot  be  assured  that 
its  efforts  are  effectively  and  efficiently  addressing  stakeholders’ 
concerns.  Together,  not  conducting  the  additional  analyses  to  improve 
data  reliability  and  leaving  out  some  content  in  its  surveys  may  result  in 
FVAP  obtaining  incomplete  evaluative  information  from  its 
postelection  surveys. 

•  Measures  of  success.  The  measures  of  success  include  the  number  of 
VAOs  trained,  number  of  Federal  Post  Card  Applications  and  Voting 
Assistance  Guides  distributed,  and  number  of  times  voting  messages 
were  included  on  leave  and  earning  statements,  among  other  measures. 
Such  measures  are  one  type  of  information  that  FVAP  uses  to  assess 
two  requirements  identified  in  the  DOD  directive:  (1)  ensure  voters 
receive  information  about  registration,  voting  procedures,  and 
materials  pertaining  to  upcoming  elections  and  (2)  conduct  voting 
assistance  workshops  for  VAOs  in  even-numbered  years  worldwide. 
While  the  measures  of  success  are  credible  metrics  for  evaluating  some 
of  FVAP’s  efforts,  FVAP  did  not  conduct  a  reliability  assessment  of  the 
data  for  the  measures  before  the  DOD  Office  of  Inspector  General  used 
the  information  in  the  2009  report.  In  fact,  we  found  that  some  of  the 
measures  may  be  artificially  high  or  low,  indicating  reliability  concerns. 
For  example,  because  FVAP  uses  multiple  methods  to  deliver  the 
Federal  Post  Card  Application  for  absentee  ballots,  asking  VAOs  to 
report  the  number  of  copies  distributed  could  result  in  (1)  over¬ 
counting  of  servicemembers  if  they  received  multiple  copies  of  the  post 
card  through  multiple  outreach  efforts  or  (2)  under-counting  of 
servicemembers  if  some  servicemembers  only  obtained  a  copy  of  the 
post  card  from  FVAP’s  Web  site.  We  identified  additional 
inconsistencies  in  the  FVAP-reported  data,  such  as  one  service’s  total 
number  of  trained  VAOs  was  reported  as  approximately  2,500  persons 
greater  than  its  total  number  of  VAOs.  Also,  even  though  the  data  are 
self-reported,  FVAP  did  not  instruct  either  the  VAOs  or  the  services 
how  to  collect  the  evaluative  information  in  order  to  promote  data 
consistency.  In  addition,  while  these  measures  were  developed  in 
concert  with  the  services,  they  were  developed  according  to  the  data 
that  were  readily  available  to  the  services,  rather  than  using  established 
guidance  on  developing  performance  measures  to  thoroughly  assess 
FVAP’s  effectiveness  and  efficiency.  Due  to  the  lack  of  data 
verification,  the  inconsistencies  we  identified  in  the  data  submitted  to 
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the  DOD  Office  of  Inspector  General,  and  credibility  concerns,  the 
extent  to  which  the  data  accurately  measure  the  effectiveness  or 
efficiency  of  FVAP’s  efforts  is  unknown. 


When  we  discussed  FVAP’s  evaluative  information  with  FVAP’s  new 
Director,  he  noted  that  he  intends  to  continue  improving  the  methods  that 
FVAP  uses  to  gather  evaluative  information.  For  example,  he  is  reaching 
out  to  obtain  evaluative  assistance  both  within  DOD  and  from  outside 
sources.  He  also  indicated  that  he  intends  to  add  performance  measures  to 
examine  the  impact  of  FVAP’s  efforts  and  supplement  the  evaluative 
information  (e.g.,  numbers  of  materials  distributed  and  VAOs  trained)  that 
the  agency  currently  gathers  on  its  processes. 

We  also  found  that  FVAP  is  not  systematically  collecting  evaluative 
information  on  all  aspects  of  its  VAO  training  program.  While  VAO 
outreach  to  potential  voters  is  integral  to  several  of  FVAP’s  efforts  to 
achieve  its  mission,  FVAP  has  not  formally  evaluated  its  in-person  or 
online  training  course  that  each  VAO  receives.  FVAP  does,  however, 
obtain  ad-hoc  feedback  from  VAOs  while  conducting  its  in-person  training 
workshops  and  through  a  general  feedback  question  on  the  postelection 
survey  to  VAOs.  In  a  2004  report,  we  noted  that  training  programs  should 
have  an  evaluative  component  to  help  ensure  the  training  is  effective  and 
improves  performance  and  results.  We  also  stated  that  agencies  need 
evaluation  processes  that  systematically  track  the  cost  and  delivery  of 
training. 11  The  cost  associated  with  completing  the  online  VAO  training  is 
primarily  indirect  (e.g.,  personnel  time  to  complete  the  course).  In 
contrast,  in-person  VAO  training  results  in  travel  costs  for  the  FVAP 
trainers  and  possibly  some  VAOs  who  must  travel  to  another  location  to  be 
trained.  The  locations  of  the  2-hour  in-person  VAO  training  further 
illustrate  the  higher  cost  for  the  in-person  training.  In  2008,  FVAP  offered 
in-person  VAO  training  workshops  in  a  variety  of  locations:  20  in  the  area 
around  Washington,  D.C.,  28  in  other  U.S.  locations,  and  62  in  foreign 
locations.  Without  a  formal  evaluation  of  the  training,  it  is  not  possible  to 
know  (1)  how  well  either  training  method  conveys  knowledge  to  VAOs,  (2) 
steps  needed  to  enhance  the  training,  (3)  whether  in-person  training 
provides  added  benefits  to  outweigh  its  additional  costs,  and  (4)  whether 
new  technology — such  as  Webinars — could  result  in  more  effective, 
efficient,  or  timely  delivery  of  the  training.  Since  the  VAO  training  is  given 


GAO,  Human  Capital:  A  Guide  for  Assessing  Strategic  Training  and  Developmental 
Efforts  in  the  Federal  Government,  GAO-04-546G  (Washington,  D.C.:  March  2004). 
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only  in  even  numbered  years,  delaying  a  formal  evaluation  of  the  two 
methods  of  training  until  2012  would  mean  that  there  would  be  no 
empirical  basis  for  adjusting  the  training  until  2014. 

In  discussing  potential  reasons  why  some  of  the  concerns  about  the 
evaluative  information  remain,  FVAP’s  new  Director  said  that  these  types 
of  concerns  exist  in  many  organizations  where  the  past  emphasis  focuses 
only  on  whether  something  was  implemented,  rather  than  on  impact. 
However,  in  later  discussions,  FVAP's  new  Director  mentioned  that  the 
agency  will  perform  more  reviews  of  evaluative  information  in  the  future. 
Continued  use  of  the  evaluative  information  without  adding  steps  such  as 
prescribing  procedures  for  the  self-reporting  of  information  and 
conducting  non-response  analyses  could  limit  (1)  FVAP’s  ability  to 
enhance  the  effectiveness  and  efficiency  of  its  overall  program  and  (2) 
DOD’s  ability  to  provide  a  more  complete  picture  of  FVAP’s  performance, 
which  could  impact  the  quality  of  future  reports  to  Congress  submitted  in 
response  to  section  586  of  the  National  Defense  Authorization  Act  for 
Fiscal  Year  2010. 12 


FVAP’s  Budget 
Priorities  Appear  to 
Be  Generally  Aligned 
with  Its  Strategic  Plan 
Goals 


While  we  could  not  perform  detailed  analyses  because  FVAP  does  not 
budget  by  strategic  goals,  FVAP’s  budgets  for  fiscal  years  2005  through 
2009  and  the  goals  in  its  strategic  plans  for  the  same  period  appear  to  be 
generally  aligned.  FVAP  uses  major  budget  categories  and  budget 
directions  from  the  Defense  Human  Resource  Agency.  FVAP’s  budget 
averaged  approximately  $4  million  for  the  5  years.  Table  2  shows  the 
amounts  for  each  fiscal  year  and  the  average  across  the  5  years  for  each  of 
six  major  budget  categories. 


12  National  Defense  Authorization  Act  for  Fiscal  Year  2010,  Pub.  L.  No.  111-84,  §586  (2009), 
requires  the  presidential  designee  to  submit  an  annual  report  to  the  President  and  to  the 
relevant  committees  of  Congress.  That  report  is  to  include  an  assessment  of  the 
effectiveness  of  activities  carried  out  under  42  U.S.C.  §1973ff-2B,  including  the  activities 
and  actions  of  FVAP,  a  separate  assessment  of  voter  registration  and  participation  by 
overseas  voters  who  are  not  members  of  the  uniformed  services,  and  a  description  of  the 
cooperation  between  states  and  federal  governments  in  carrying  out  section  1973ff-2B. 
Additionally,  the  report  is  to  contain  a  description  of  the  utilization  of  voter  registration 
assistance  under  10  U.S.C.  §1533a,  including  a  description  of  the  specific  programs 
implemented  by  each  military  department  of  the  Armed  Forces  pursuant  to  section  1533a 
and  the  number  of  absent  uniformed  services  voters  who  utilized  voter  registration 
assistance  provided  under  that  section. 


Page  10 


GAO-10-476  Elections 


Table  2:  FVAP’s  Budgets  for  Fiscal  Years  2005  through  2009  Broken  into  Six  Major  Categories 


Budget  category 

Average” 

Fiscal  year 

2005 

2006 

2007 

2008 

2009 

Personnel 

$1,822,000 

$1,672,000 

$1,710,000 

$1,800,000 

$1,727,000 

$2,200,000 

Contracts 

1,665,000 

1,787,000 

1,497,000 

1,642,000 

2,306,000 

1,091,000 

Operations 

234,000 

285,000 

231,000 

176,000 

158,000 

322,000 

Travel 

145,000 

35,000 

220,000 

60,000 

350,000 

60,000 

Information  technology 

103,000 

55,000 

125,000 

78,000 

105,000 

150,000 

Training 

33,000 

25,000 

14,000 

15,000 

50,000 

60,000 

Total 

$4,001,000 

$3,859,000 

$3,797,000 

$3,771,000 

$4,696,000 

$3,883,000 

Source: 

GAO  analysis  of  FVAP  data. 

Note:  T otals  may  not  include  congressional  adjustments. 

The  averages  in  this  column  are  rounded  to  the  nearest  thousand  dollars. 

According  to  FVAP’s  budget  requests  to  Congress  for  fiscal  year  2009,  the 
program’s  overall  budget  generally  supports  the  achievement  of  its 
mission  and  strategic  goals.  The  strategic  plan  for  2008  and  2009  contained 
the  following  four  general  goals:  (1)  encourage  adoption  of  FVAP 
legislative  initiatives  in  order  to  facilitate  UOCAVA  voting,  (2)  maximize 
use  of  existing  and  emerging  technologies,  (3)  improve  marketing  and 
outreach  efforts  with  customers  and  stakeholders,  and  (4)  enhance 
resourcing  of  FVAP  programs. 

FVAP’s  travel  budget  varies  in  election  years  versus  non-election  years, 
and  the  increased  budgets  in  election  years  are  related  to  travel  associated 
with  in-person  training.  In  the  non-election  years  of  2005,  2007,  and  2009 
when  FVAP  staff  members  conducted  very  few  VAO  workshops,  the 
amounts  budgeted  for  travel  in  those  fiscal  years  were  $35,000,  $60,000, 
and  $60,000,  respectively.  FVAP  officials  told  us  that  the  travel  budget 
during  an  odd-numbered  fiscal  year  consists  primarily  of  staff  members’ 
trips  to  state  conferences  to  meet  with  state  and  local  election  officials. 
Such  efforts  generally  correspond  to  FVAP’s  goal  1  objective  of  engaging 
the  states  and  territories  in  an  effort  to  pass  recommended  legislative 
initiatives.  In  contrast,  the  travel  budgets  for  fiscal  years  2006  and  2008 
were  much  higher — $220,000  and  $350,000,  respectively.  The  higher 
amounts  reflect  FVAP’s  traveling  for  the  previously  mentioned  meetings 
with  election  officials  and  to  locations  in  the  U.S.  and  foreign  countries  in 
order  to  conduct  VAO  training  workshops  during  the  fiscal  years  that 
directly  precede  federal  elections.  This  latter  purpose  corresponds  to  the 
goal  2  objective  to  enhance  training  products  and  services. 
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Our  analysis  of  other  documents  found  that  FVAP’s  budget  for  contracts 
also  appears  to  contribute  to  the  agency’s  efforts  to  achieve  its  strategic 
goals  and  objectives.  Funding  in  this  category  is  particularly  important  in 
supporting  goal  2  and  goal  3  objectives.  We  will  use  the  fiscal  year  2008 
contracts  budget  to  illustrate  some  linkages  between  FVAP’s  budget  and 
its  strategic  goals.  For  example, 

•  Some  of  FVAP’s  contract  funds  were  linked  to  goal  2  objectives  such  as 
improving  the  Web  site  and  exploring  new  technology  methods  for 
Internet  voting.  For  example,  in  2008  FVAP  used  funds  from  their 
contracts  budget  for  the  Electronic  Transmission  Service, 13  absentee 
ballot  tracking  equipment  for  the  Military  Postal  Service  Agency,  an 
electronic  voter  registration  and  ballot  delivery  system,  and  redesign  of 
the  FVAP  Web  site.  These  services  and  technologies  are  used  to  reach 
UOCAVA  voters  and  state  and  local  election  officials,  FVAP’s  target 
populations.  FVAP  officials  told  us  that  some  expenses,  such  as  funds 
spent  on  the  Electronic  Transmission  Service  and  publication  printing 
(e.g.,  the  Voting  Assistance  Guide,  motivational  posters,  and  election 
dates  posters),  reoccurred  between  fiscal  years  2005  through  2009. 

•  Funds  spent  from  the  contracts  budget  related  to  the  postelection 
surveys  generally  link  to  goal  2  and  3  and  involve  the  use  of  existing 
and  emerging  technologies  and  the  improvement  of  the  program’s 
marketing  and  outreach  efforts  with  stakeholders.  According  to  FVAP 
officials,  these  initiatives  aid  in  the  design  and  execution  of  FVAP’s 
postelection  surveys.  Resources  for  postelection  survey  planning  are 
required  in  the  odd-numbered  fiscal  year  prior  to  a  presidential  election 
and  additional  resources  for  planning  and  executing  the  survey  and 
analyzing  the  results  continue  into  at  least  the  next  2  fiscal  years. 


Although  it  was  not  possible  for  us  to  determine  how  much  of  the 
personnel  budget  could  be  linked  directly  to  a  specific  strategic  goal,  our 
review  found  that  accomplishment  of  each  of  FVAP’s  strategic  objectives 
required  actions  by  the  agency’s  staff.  Among  other  things,  the  first  goal  of 
encouraging  adoption  of  FVAP  legislative  initiatives  is  accomplished  by 
FVAP  staff  developing  the  initiatives  that  will  facilitate  UOCAVA-related 
voting  and  coordinating  with  state  and  local  election  officials  to  accept  the 


13  The  Electronic  Transmission  Service  is  a  resource  that  allows  fax-to-email  and  email-to- 
fax  conversion  of  election  materials  during  transmission  between  state  and  local  election 
officials  and  UOCAVA  voters. 
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initiatives.  Moreover,  the  goal  4  objective  that  is  related  to  employing 
improved  budgeting,  staffing,  and  training  is  accomplished  by  FVAP  staff. 

FVAP  officials  told  us  that  the  remaining  three  budget  categories — 
operations,  information  technology,  and  training  for  FVAP  employees — 
support  FVAP’s  daily  operations.  The  operations  budget  includes  the  cost 
of  utilities  for  the  program’s  office  space,  supplies,  and  forms  purchases 
(e.g.,  Federal  Write-in  Absentee  Ballot  and  Federal  Post  Card  Application). 
Information  technology  includes  basic  Web  site  expenses  and  computer 
equipment  purchases.  Our  analysis  and  discussion  with  FVAP  officials 
revealed  that  these  budget  categories  provide  resources  for  FVAP  staff  to 
perform  their  work  which  helps  them  to  meet  FVAP’s  strategic  goals. 


FVAP  and  USD  (P&R) 
Fully  or  Partially 
Implemented  Most  of 
the  Recommendations 


FVAP  and  USD  (P&R)  fully  or  partially  implemented  17  of  23  DOD  Office 
of  Inspector  General  and  GAO  recommendations  made  from  2001  through 
2009. 14  The  23  recommendations  involve  improvements  in  the  (1)  oversight 
and  direction  FVAP  and  USD  (P&R)  provide  to  the  services’  respective 
voting  assistance  programs  through  formal  policy  and  implementation 
guidance  and  (2)  processes  for  carrying  out  FVAP’s  mission.  FVAP  and 
USD  (P&R)  fully  implemented  8  of  16  DOD  Office  of  Inspector  General 
and  GAO  recommendations  for  improving  the  oversight  and  direction  that 
FVAP  provides  to  the  services  and  partially  implemented  three  of  the  other 
eight  recommendations.  Additionally,  FVAP  fully  implemented  three  GAO 
recommendations  for  improvements  in  the  processes  it  uses  to  carry  out 
its  mission  and  partially  implemented  three  of  the  four  remaining 
recommendations.  Table  3  summarizes  FVAP’s  and  USD  (P&R)’s 
implementation  of  DOD  Office  of  Inspector  General’s  and  GAO’s 
recommendations. 


14  See  appendix  II  for  DOD  Office  of  Inspector  General  and  GAO  reports  reviewed  from 
2001  to  2009  related  to  these  recommendations. 
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Table  3:  FVAP’s  and  USD  (P&R)’s  Implementation  of  DOD  Office  of  Inspector  General  and  GAO  Recommendations:  2001 
through  2009 

Area  of  improvement 

Source  of  recommendation  and  recommendation 
status 

Oversight  and  direction 
to  the  services 

FVAP  processes 

Total 

DOD  Office  of  Inspector  General  recommendations: 

Implemented 

5 

0 

5 

Partially  implemented 

2 

1 

3 

Not  implemented 

5 

1 

6 

Subtotal:  DOD  Inspector  General 

12 

2 

14 

GAO  recommendations: 

Implemented 

3 

3 

6 

Partially  implemented 

1 

2 

3 

Not  implemented 

0 

0 

0 

Subtotal:  GAO 

4 

5 

9 

Combined  DOD  Inspector  General  and  GAO 
recommendations 


Implemented  8 

3 

11 

Partially  implemented  3 

3 

6 

Not  implemented  5 

1 

6 

Total:  Combined  DOD  Inspector  General  and  GAO  16 

7 

23 

Source:  GAO  analysis  of  DOD  Office  of  Inspector  General,  FVAP,  and  GAO  data. 

FVAP  and  USD  (P&R)  fully  implemented  8  of  16  DOD  Office  of  Inspector 
General  and  GAO  recommendations  and  partially  implemented  3 
recommendations  to  improve  the  oversight  and  direction  FVAP  provides 
to  the  services  to  guide  their  implementation  of  service-  and  installation- 
level  voting  assistance  programs.  However,  5  of  the  16  recommendations 
have  not  been  implemented.  In  recent  years,  several  revisions  have  been 
made  to  DOD  Directive  1000.04,  Federal  Voting  Assistance  Program 
(FVAP),  and  many  of  the  changes  are  related  to  DOD  Office  of  Inspector 
General  and  GAO  recommendations.  For  example,  the  directive  was 
revised  in  2002  to  direct  the  services  to  develop  written  policies  to  support 
all  eligible  military  personnel  and  their  family  members,  following  a  2001 
GAO  recommendation  that  DOD  direct  service  secretaries  to  develop 
voting  assistance  directives  that  fully  reflect  the  requirements  of  DOD 
Directive  1000.04.  DOD  Directive  1000.04  was  also  revised  in  2002  to 


FVAP  and  USD  (P&R) 
Fully  or  Partially 
Implemented  Most  of  the 
Recommendations  to 
Improve  Oversight  and 
Direction 
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require  the  heads  of  DOD  components  to  assign  one  Unit  VAO  within  each 
unit  of  25  or  more  permanently  assigned  members,  and  was  revised  again 
in  2004  to  require  assignment  of  an  additional  Unit  VAO  for  each 
additional  50  members  above  the  25  member  base.  Prior  to  these 
revisions,  we  reported  that  VAOs  were  sometimes  responsible  for  too 
many  voters  as  a  result  of  the  lack  of  DOD-wide  criteria  for  determining 
how  many  voters  a  VAO  should  serve  and  recommended  that  DOD  consult 
with  the  services  and  revise  DOD  Directive  1000.04  to  establish 
recommended  ratios  of  VAOs  to  population  served.  In  addition  to  changes 
made  through  the  directive,  FVAP  and  USD  (P&R)  directed  the  revision  of 
service-level  voting  assistance  regulations  and  provided  additional 
clarifying  guidance  to  the  military  services  through  the  biennial  FVAP 
voting  action  plans  and  memorandums.  Table  4  summarizes  the  DOD 
Office  of  Inspector  General’s  and  GAO’s  recommendations  to  improve 
oversight  of  and  direction  to  the  services  and  lists  the  actions  that  FVAP 
and  USD  (P&R)  took  to  implement  the  recommendations. 


Table  4:  DOD  Office  of  Inspector  General  and  GAO  Recommendations  to  Improve  Oversight  of  and  Direction  to  the  Services 
and  FVAP’s  and  USD  (P&R)’s  Actions  Taken  to  Implement  the  Recommendations:  2001  through  2009 

Recommendation  (source,  year) 

Actions  taken 

Direct  service  secretaries  to  develop  voting  assistance 
directives  that  fully  reflect  the  requirements  of  DOD  Directive 
1000.04.  (GAO,  2001) 

Implemented:  USD  (P&R)  revised  Directive  1000.04  in  2002  to  direct 
the  services  to  develop  written  policies  to  support  all  eligible  military 
personnel  and  their  family  members. 

Consult  with  the  services  and  revise  DOD  Directive  1000.04 
to  establish  recommended  ratios  of  VAOs  to  population 
served.  (GAO,  2001) 

Implemented:  USD  (P&R)  revised  Directive  1000.04  in  2002  to  require 
thatthe  heads  of  DOD  components  assign  one  unit  VAO  within  each 
unit  of  25  or  more  permanently  assigned  members,  and  added 
language  in  2004  stating  thatan  additional  unitVAO  should  be 
assigned  for  each  additional  50  members  above  the  25  member  base. 

Actively  collect  and  share  best  practices  identified  by  service 
and  Department  of  State  voting  assistance  programs.  (GAO, 
2001) 

Implemented:  DOD's  biennial  voting  action  plans  require  FVAP  to 
collect  and  share  best  practices  identified  by  the  services  and 
Department  of  State.  FVAP  published  best  practices  on  its  Web  site 
and  in  the  Voting  Information  News  newsletter  that  it  distributes 
monthly  to  VAOs,  and  discussed  them  at  voting  workshops. 

Direct  service  secretaries  to  require  that  the  senior  service 
voting  representatives  monitor  installations' voting  assistance 
programs,  periodically  provide  briefings  to  FVAP  on  the 
services'  efforts  to  comply  with  DOD  policy,  and  submit  a  final 
report  byj  une  30  of  odd-numbered  years.  (GAO,  2001) 

Partially  implemented:  DOD  Directive  1000.04  requires  thatthe 
services  file  an  after-action  report  to  FVAP,  and  the  biennial  voting 
action  plans  require  thatthese  reports  be  filed  byj  anuary  15  of  odd- 
numbered  years.  In  late  2003,  the  USD  (P&R)  began  holding  monthly 
meetings  with  FVAP  and  the  senior  service  voting  representatives  to 
discuss  the  status  of  service  voting  assistance  programs.  Neitherthe 
DOD  Directive  nor  the  voting  action  plans  explicitly  require  the  senior 
service  voting  representatives  to  monitor  installation-level  voting 
assistance  programs. 
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Recommendation  (source,  year) 

Actions  taken 

Provide  oversight  to  ensure  that  service  voting  assistance 
program  regulations  are  consistent  with  DOD  Directive 

1000.04  requirements.  (DOD  Office  of  Inspector  General, 
2003) 

Implemented:  According  to  follow  up  completed  by  the  Office  of 
Inspector  General,  USD  (P&R)  fulfilled  this  recommendation  by  issuing 
a  memorandum  to  require  thatthe  service  Secretaries  revise  their 
policies  to  correct  discrepancies  DOD  Office  of  Inspector  General 
found  between  the  policies  and  DOD  Directive  1000.04. 

Expedite  the  revision  and  issuance  of  DOD  Directive  1000.04 
and  the  DOD  Voting  Action  Plan  for  2004-2005.  (DOD  Office 
of  Inspector  General,  2004) 

Implemented:  DOD  issued  the  revised  Directive  1000.04  on  April  14, 
2004.  USD  (P&R)  and  FVAP  released  the  Voting  Action  Plan  for  2004- 
2005  on  February  11,  2004. 

Provide  oversight  to  ensure  that  service  voting  assistance 
program  regulations  and  service  voting  plans  are  consistent 
with  the  requirements  established  by  the  new  guidance. 

(DOD  Office  of  Inspector  General,  2004) 

Implemented:  FVAP  reviewed  the  revised  service  policies  to  ensure 
compliance  with  DOD  Directive  1000.04. 

Revise  DOD  Directive  1000.04  to  require  FVAP  and  the 
services  to  collect  and  analyze  metrics  on  a  more  frequent 
basis.  (DOD  Office  of  Inspector  General,  2005) 

Not  implemented:  The  current  version  of  the  draft  DOD  Instruction 
1000.04,  if  finalized,  will  require  the  director  of  FVAP  to  establish 
minimum  voting  program  metrics  to  be  used  by  the  DOD  components 
and  the  Uniformed  Services  in  evaluating  their  individual  voting 
assistance  programs,  and  report  on  the  results  of  those  metrics.3 

Revise  DOD  Directive  1000.04  to  require  thatVAO  training 
include  all  program  objectives  outlined  in  DOD  Directive 
1000.04  and  good  ideas  and  best  practices  as  tools  for 
accomplishing  objectives,  and  thatVAOs  should  complete 
FVAP  training  within  60  days  of  their  appointment.  (DOD 

Office  of  Inspector  General,  2005) 

Not  implemented:  FVAP  collects  and  shares  best  practices,  and 
discusses  them  atVAO  training  workshops.  Flowever,  based  on  our 
observation,  VAO  training  does  not  address  all  program  objectives  of 
DOD  Directive  1000.04,  and  the  Directive  does  not  require  that  all 
objectives  be  addressed  in  the  training.  The  current  version  of  draft 

DOD  Instruction  1000.04,  if  finalized,  would  require  VAOs  to  complete 
training  within  90  days  of  their  appointment.3 

Revise  DOD  Directive  1000.04  to  permit  heads  of  the  DOD 
components  and  the  services  to  exercise  flexibility  in  how 
many  unit  VAOs  they  assign  and  at  what  level.  (DOD  Office 
of  Inspector  General,  2006) 

Not  implemented:  USD  (P&R)  issued  a  memorandum  to  the 

Secretaries  of  the  Military  Departments  stating  thatthe  guidance  and 
direction  found  in  DOD  Directive  1000.04,  which  includes  guidance 
regarding  the  ratio  of  VAOs  to  assigned  members  within  a  unit  and  the 
desired  rank  or  pay  grade  of  VAOs,  should  be  used  as  a  baseline  and 
may  be  adjusted  to  meet  specific  requirements  and  cultures  within 
each  service.  In  addition,  the  memorandum  describes  revisions  to  the 
relevant  provisions  in  the  current  version  of  the  draft  DOD  Instruction 
1000.04  and  emphasizes  the  flexibility  of  those  provisions.3 

Direct  the  services  to  establish  a  "one-click"  Web  site  access 
from  its  home  page  to  a  voting  page  that  provides  service  and 
FVAP -related  information  and  materials.  (DOD  Office  of 
Inspector  General,  2006) 

Partially  implemented:  The  current  version  of  draft  DOD  Instruction 
1000.04,  if  finalized,  will  require  each  service  to  have  a  direct  link  from 
its  Web  site  home  page  to  the  service's  voting  assistance  program 
home  page. 

Publish  guidance  to  the  services  clarifying  DOD  Directive 
1000.04  on  the  commander's  flexibility  in  establishing  the 
ratio  of  unit  VAOs  to  unit  members.  (DOD  Office  of  Inspector 
General,  2007) 

Implemented:  USD  (P&R)  issued  a  memorandum  to  the  Secretaries  of 
the  Military  Departments  stating  thatthe  guidance  and  direction  found 
in  DOD  Directive  1000.04,  which  includes  guidance  regarding  the  ratio 
of  VAOs  to  assigned  members  within  a  unit,  should  be  used  as  a 
baseline  and  may  be  adjusted  to  meet  specific  requirements  and 
cultures  within  each  service.  In  addition,  the  memorandum  describes 
revisions  to  the  relevant  provisions  in  the  current  version  of  the  draft 
DOD  Instruction  1000.04  and  emphasizes  the  flexibility  of  those 
provisions.3 
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Recommendation  (source,  year) 

Actions  taken 

Revise  DOD  Directive  1000.04  to  provide  flexibility  for  pay 
grades  required  of  personnel  assigned  VAO  duties.  (DOD  IG, 
2007) 

Not  implemented:  USD  (P&R)  issued  a  memorandum  to  the 

Secretaries  of  the  Military  Departments  stating  thatthe  guidance  and 
direction  found  in  DOD  Directive  1000.04,  which  includes  guidance 
regarding  the  desired  rank  or  pay  grade  of  VAOs,  should  be  used  as  a 
baseline  and  may  be  adjusted  to  meet  specific  requirements  and 
cultures  within  each  service.  In  addition,  the  memorandum  describes 
revisions  to  the  relevant  provisions  in  the  current  version  of  the  draft 
DOD  Instruction  1000.04,  and  emphasizes  the  flexibility  of  those 
provisions.3 

Publish  guidance  to  the  services  emphasizing  the  importance 
and  benefits  of  appointing  civilian  personnel  as  installation 
VAOs.  (DOD  Office  of  Inspector  General,  2007) 

Implemented:  USD  (P&R)  issued  a  memorandum  to  the  Secretaries  of 
the  Military  Departments  stating  thatthe  guidance  and  direction  found 
in  DOD  Directive  1000.04,  which  says  thatwhere  possible,  Installation 
VAOs  should  be  a  civilian  GS-12  or  higher,  should  be  used  as  a 
baseline  and  may  be  adjusted  to  meet  specific  requirements  and 
cultures  within  each  service.  In  addition,  the  memorandum  describes 
revisions  to  the  relevant  provisions  in  the  current  version  of  the  draft 
DOD  Instruction  1000. 04. a 

Direct  the  services  to  comply  with  a  recommendation  from  the 
2005  report  to  establish  one-click  links  from  their  home  page 
to  its  voting  assistance  site.  (DOD  Office  of  Inspector 

General,  2007) 

Partially  implemented:  The  current  version  of  draft  DOD  Instruction 
1000.04,  if  finalized,  will  require  each  service  to  have  a  direct  link  from 
its  Web  site  home  page  to  the  service's  voting  assistance  program 
home  page. 

Revise  DOD  Directive  1000.04  to  authorize  appointment  of 
civilian  personnel  as  unitVAOs  where  feasible.  (DOD  Office 
of  Inspector  General,  2007) 

Not  implemented:  USD  (P&R)  determined  thatthis  recommendation 
may  not  be  practicable  because  civilian  government  personnel  within 
the  United  States  may  not  be  authorized  by  law  to  administer  oaths. 

Source:  GAO  analysis  of  DOD  Office  of  Inspector  General,  USD  (P&R),  FVAP,  and  GAO  information. 


“Although  the  DOD  Instruction  1000.04,  which  will  replace  the  current  DOD  Directive  1000.04,  was 
still  in  draft  during  our  review,  FVAP  officials  told  us  thatthe  services  had  been  informed  of  the  new 
provisions  in  the  policy  for  the  purpose  of  implementing  their  respective  voting  assistance  programs. 


FVAP  and  USD  (P&R)  partially  implemented  three  other 
recommendations.  In  2001,  GAO  recommended  that  the  service 
Secretaries  should  (1)  require  the  senior  service  voting  representatives  to 
monitor  installations’  voting  assistance  programs,  (2)  periodically  update 
FVAP  on  efforts  to  meet  the  requirements  of  DOD  Directive  1000.04,  and 
(3)  submit  a  final  report  by  June  30  of  odd-numbered  years.  In  response, 
USD  (P&R)  began  holding  monthly  meetings  in  late  2003  with  FVAP  staff 
and  senior  service  voting  representatives  to  discuss  the  status  of  service 
voting  assistance  programs.  However,  neither  DOD  Directive  1000.04  nor 
FVAP’s  biennial  voting  action  plans  explicitly  required  senior  service 
voting  representatives  to  monitor  installations’  voting  assistance 
programs.  In  our  2001  report,  we  noted  that  such  monitoring  would  enable 
the  services  to  identify  and  address  weaknesses  in  their  programs.  In 
addition,  FVAP  partially  addressed  a  2006  and  2007  DOD  Office  of 
Inspector  General  recommendation  in  its  the  current  draft  that  includes  a 
requirement  that  each  service  have  a  direct  link  from  its  Web  site  home 
page,  to  the  service’s  voting  assistance  program  home  page. 
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FVAP  did  not  implement  five  of  the  16  recommendations.  While  FVAP  has 
created  a  draft  version  of  DOD  Instruction  1000.04,  it  is  still  under 
development  and  has  not  been  finalized.  The  draft  of  DOD  Instruction 
1000.04 — if  finalized  as  currently  written — would  replace  the  existing 
directive  and  include  several  new  provisions  responding  to  DOD  Office  of 
Inspector  General  recommendations  that  call  for  revisions  to  Directive 
1000.04.  For  example,  the  DOD  Office  of  Inspector  General  recommended 
that  DOD  revise  Directive  1000.04  to  require  FVAP  and  the  services  to 
collect  and  analyze  metrics  on  a  more  frequent  basis.  While  FVAP  and  the 
services  recently  developed  metrics,  and  the  draft  DOD  Instruction,  if 
finalized,  would  require  the  director  of  FVAP  to  (1)  establish  minimum 
voting  program  metrics  to  be  used  by  the  DOD  components  and  the 
Uniformed  Services  in  evaluating  their  individual  voting  assistance 
programs,  and  (2)  report  on  the  results  of  those  metrics,  the  draft  guidance 
has  not  been  finalized  and  the  directive  has  not  been  revised  as 
recommended.  In  addition,  recommendations  were  made  to  revise  DOD 
Directive  1000.04  to  (1)  require  that  VAO  training  include  all  program 
objectives,  (2)  permit  heads  of  the  DOD  components  and  the  services  to 
exercise  flexibility  in  how  many  unit  VAOs  they  assign,  and  (3)  provide 
more  flexibility  for  pay  grades  required  of  personnel  assigned  VAO  duties. 
FVAP  also  did  not  implement  a  2005  DOD  Office  of  Inspector  General 
recommendation  to  revise  DOD  Directive  1000.04  to  require  that  VAO 
training  cover  all  program  objectives  in  the  directive  and  best  practices, 
and  that  the  training  be  completed  within  60  days  of  a  VAO’s  appointment. 
The  current  draft  DOD  Instruction  1000.04  if  finalized  will  require  that 
VAO  training  be  completed  within  90  days  of  an  individual’s  appointment. 
Further,  while  FVAP  does  collect  best  practices  among  VAOs  and  shares 
some  of  these  in  VAO  training  workshops,  the  training  workshops  we 
observed  did  not  cover  all  the  program  objectives  from  DOD  Directive 
1000.04.  For  example,  the  workshops  did  not  cover  the  requirement  that 
the  DOD  components  and  uniformed  services  are  to  establish  and 
publicize  a  special  telephone  service,  a  “Voting  Action  Line,”  to  link  unit 
voting  officers  with  their  respective  Service  or  Departmental  Voting  Action 
Officer.  In  addition,  FVAP  did  not  implement  a  2007  DOD  Office  of 
Inspector  General  recommendation  to  revise  Directive  1000.04  to 
authorize  appointment  of  civilian  personnel  as  unit  VAOs  in  order  to 
increase  continuity  in  the  unit  VAO  positions.13  In  the  written  response  to 
this  recommendation,  USD  (P&R)  stated  that  this  recommendation  may 


15  In  general,  civilians  serve  for  a  longer  period  of  time  than  servicemembers  before  moving 
to  a  different  unit. 
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not  be  practicable.  By  way  of  explanation,  it  cited  potential  limitations  on 
the  authority  of  civilian  government  employees  within  the  United  States  to 
administer  oaths. 


FVAP  Fully  or  Partially 
Implemented  All  But  One 
Recommendation  to 
Improve  Its  Processes 


FVAP  fully  implemented  three  of  seven  recommendations  pertaining  to 
changes  in  the  processes  that  it  uses  to  execute  its  mission,  and  it  partially 
implemented  another  three  such  recommendations.  For  example,  in  2001 
we  recommended  that  FVAP  revise  its  postelection  survey  methodology  to 
gather  nationally  projectable  data  on  disqualified  overseas  and  military 
absentee  ballots  from  local  election  officials.  In  response,  FVAP 
redesigned  the  postelection  survey  to  obtain  the  number  of  absentee 
ballots  from  UOCAVA-eligible  voters  not  counted  and  the  reasons  for 
disqualifying  the  ballots.  At  the  time  of  our  recommendation,  we 
concluded  that  the  collection  and  analysis  of  this  information  over  time 
could  be  useful  in  helping  assess  whether  efforts  to  improve  voting 
assistance  and  work  with  states  have  a  positive  impact  on  military  and 
overseas  citizens  being  able  to  vote.  Moreover,  such  information  should 
help  Congress  assess  the  extent  to  which  military  and  overseas  ballots  are 
disqualified  and  evaluate  whether  a  legislative  remedy  is  needed.  FVAP 
also  took  steps  to  implement  two  GAO  recommendations  from  2007  to 
improve  the  security  and  accuracy  of  its  electronic  and  Internet  initiatives. 
The  steps  included  (1)  the  addition  of  a  cautionary  statement  to  FVAP’s 
electronic  blank  ballot  request  and  delivery  system  to  warn  UOCAVA 
voters  to  remove  personal  data  from  their  computers  and  (2)  the 
development  of  a  process  to  review  online  UOCAVA  voting  guidance  at  the 
FVAP  Web  site  by  creating  a  checklist  with  steps  that  FVAP  staff  must 
take  when  making  changes  to  the  Web  site.  Table  5  summarizes  the  DOD 
Office  of  Inspector  General’s  and  GAO’s  recommendations  to  improve 
FVAP  processes  and  lists  the  actions  taken  by  FVAP  to  implement  the 
recommendations. 
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Table  5:  DOD  Office  of  Inspector  General  and  GAO  Recommendations  to  Improve  FVAP  Processes  and  Actions  Taken  by 
FVAP  to  Implement  the  Recommendations:  2001  through  2009 


Recommendation  (source,  year) 

Action  taken 

Develop  a  methodology  to  gather  nationally  projectable  data 
on  disqualified  military  and  overseas  absentee  ballots  and 
reasons  for  disqualification.  (GAO,  2001) 

Implemented:  FVAP  redesigned  the  methodology  of  its  postelection 
survey  of  local  election  officials  to  obtain  nationally  projectable  data  on 
the  number  of  absentee  ballots  not  counted  and  the  reasons  for  the 
disqualification  of  absentee  ballots  from  military  and  overseas  citizens. 

Develop  an  automated  electronic  delivery  and  reporting 
system  that  would  be  used  to  disseminate  information  to 
voters.  (DOD  Office  of  Inspector  General,  2005) 

Not  implemented:  According  to  FVAP  officials,  they  determined,  along 
with  the  services,  that  it  was  infeasible  because  some  servicemembers 
do  not  have  e-mail  or  Internet  access. 

Incorporate  lessons  learned  into  plans  for  future  systems 
such  as  those  we  identified,  including  adding  cautionary 
statements  to  future  ballot  request  and  receipt  systems  to 
warn  UOCAVA  voters  to  remove  personal  data  from  their 
computers.  (GAO,  2007) 

Implemented:  FVAP  added  a  cautionary  statement  to  its  electronic 
blank  ballot  request  and  delivery  system. 

Institutionalize  a  process  to  review  online  UOCAVA  guidance 
to  ensure  that  DOD  provides  accurate  and  consistent 
information  to  UOCAVA  voters.  (GAO,  2007) 

Implemented:  FVAP  has  developed  a  process  to  review  online  voting 
guidance  at  its  Web  site  by  creating  a  checklist  with  steps  that  FVAP 
staff  must  take  when  changes  to  the  Web  site  are  made. 

Comply  with  the  information  security  requirements  in  the 

DOD  Certification  and  Accreditation  Process  guidance. 

(GAO,  2007) 

Partially  implemented:  As  of  February  2010,  FVAP  officials  told  us  they 
plan  to  ensure  that  the  Electronic  Transmission  Service  for  the 
November  2010  federal  election  complies  with  the  DOD  requirements. 

A  contract  for  the  service  has  not  yet  been  awarded. 

Create  an  integrated,  comprehensive,  long-term,  results- 
oriented  plan  for  future  electronic  voting  programs.  (GAO, 
2007) 

Partially  implemented:  To  date,  FVAP  has  developed  some  high-level 
tasks  and  estimated  timeframes  for  such  a  program,  but  FVAP  officials 
told  us  they  are  waiting  for  the  Election  Assistance  Commission  to 
issue  electronic  absentee  voting  guidelines  in  order  to  finish  their  plan. 

Develop  metrics  and  standards  for  assessing  effectiveness 
and  compliance  of  services'  and  DOD's  Voting  Assistance 
Programs.  (DOD  Office  of  Inspector  General,  2008) 

Partially  implemented:  FVAP  established  some  performance  metrics 
for  implementation  beginning  in  the  2008  election  year,  but  we  found 
that  some  of  these  metrics  have  weaknesses. 

Source:  GAO  analysis  of  DOD  Office  of  Inspector  General,  FVAP,  and  GAO  information. 

FVAP  partially  implemented  three  of  the  remaining  four  DOD  Office  of 
Inspector  General’s  and  GAO’s  recommendations  for  improvements  to  its 
processes.  First,  in  response  to  a  2008  DOD  Office  of  Inspector  General 
recommendation  that  USD  (P&R)  and  the  service  Secretaries  should 
develop  metrics  for  assessing  their  voting  assistance  programs,  FVAP 
established  some  performance  metrics  for  implementation  beginning  in 
the  2008  election  year.  However,  as  we  noted  earlier  in  our  report,  we 
found  that  some  of  FVAP’s  evaluative  information  can  continue  to  be 
strengthened.  Second,  FVAP  officials  told  us  they  cannot  develop  a  long¬ 
term  plan  for  future  electronic  absentee  voting  programs,  which  we 
recommended  in  2007,  without  the  establishment  of  electronic  absentee 
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voting  guidelines  by  the  Election  Assistance  Commission. 16  According  to  a 
Commission  official,  voting  guidelines  are  not  yet  complete  because, 
among  other  reasons,  National  Institute  of  Technology  officials — who  are 
providing  assistance  to  the  Commission  in  developing  the  guidelines — do 
not  believe  that  a  secure  electronic  absentee  voting  system  is 
technologically  feasible  at  this  time.  In  the  interim,  however,  FVAP  has 
established  some  preliminary,  high-level  tasks  and  estimated  time  frames 
for  the  development  of  an  electronic  absentee  voting  system.  In  our  2007 
report,  we  concluded  that  until  FVAP  develops  the  type  of  plan  we 
recommended,  DOD  is  not  in  a  position  to  address  congressional 
expectations  to  establish  secure  and  private  electronic  and  Internet-based 
voting  initiatives.  Third,  FVAP  officials  told  us  they  will  ensure  their 
Electronic  Transmission  Service  complies  with  DOD’s  information 
security  requirements — as  we  recommended  in  2007 — prior  to  the 
November  2010  federal  election.  As  we  reported  in  2008,  until  FVAP  fulfills 
this  recommendation  by  performing  and  documenting  security 
assessments  and  certifications  for  its  Electronic  Transmission  Service,  it 
has  not  taken  all  the  necessary  measures  to  secure  its  system  and  comply 
with  DOD’s  information  security  requirements. 

FVAP  did  not  implement  one  DOD  Office  of  Inspector  General 
recommendation  for  improving  its  processes.  In  2005,  DOD  Office  of 
Inspector  General  recommended  that  FVAP  develop  an  automated 
electronic  delivery  and  reporting  system  that  could  be  used  to  disseminate 
information  to  voters  via  targeted  mass  electronic  communication  and 
track  the  extent  to  which  the  information  had  reached  the  entire 
population  of  servicemembers.  FVAP  officials  told  us  they  determined  that 
this  recommendation  is  infeasible  because  not  all  servicemembers  have 
ready  access  to  e-mail  or  the  Internet.  Alternatively,  FVAP  officials  told  us 
they  have  improved  the  information  available  to  UOCAVA  voters  at  the 


lb  The  National  Defense  Authorization  Act  for  Fiscal  Year  2002,  Pub.  L.  No.  107-107,  §1604 
(2001)  directed  the  Secretary  of  Defense  to  carry  out  a  demonstration  project  under  which 
absent  uniformed  services  voters  are  permitted  to  cast  ballots  through  an  electronic  voting 
system.  Initially,  the  project  was  to  be  carried  out  in  either  the  general  election  for  federal 
office  in  either  November  2002  or  November  2004.  The  Ronald  W.  Reagan  National  Defense 
Authorization  Act  for  Fiscal  Year  2006,  Pub.  L.  No.  108-375,  §567  (2004)  amended  that 
requirement  by  requiring  the  project  to  be  implemented  during  the  first  general  election  for 
federal  office  that  occurs  after  the  Election  Assistance  Commission  establishes  electronic 
absentee  voting  guidelines  and  certifies  that  it  will  assist  the  Secretary  of  Defense  in 
carrying  out  the  project.  In  2007,  we  reported  that  DOD  had  not  moved  forward  with  the 
project  because  the  Commission  had  not  developed  the  guidelines  that  DOD  needs  to 
develop  the  demonstration  project. 
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FVAP  Web  site,  disseminated  voting  assistance  alerts  through  the  heads  of 
the  services,  and  used  the  comments  section  on  servicemembers’  leave 
and  earnings  statements  to  inform  them  about  absentee  voting  procedures. 


Conclusions 


Members  of  Congress,  advocacy  groups,  and  others  have  called  for 
improved  absentee-voting  opportunities  for  the  estimated  6-million  U.S. 
citizens  covered  by  UOCAVA  while  living  in  other  countries.  The  Secretary 
of  Defense  is  the  presidential  designee  with  the  primary  responsibility  for 
federal  UOCAVA  functions,  and  through  the  Under  Secretary  of  Defense 
for  Personnel  and  Readiness  (USD  (P&R)),  the  Federal  Voting  Assistance 
Program  provides  support  to  potential  overseas  absentee  voters  in  order 
to  facilitate  their  participation  in  federal  elections.  Toward  that  end,  FVAP 
took  many  actions  to  (1)  address  the  requirements  identified  in  DOD 
Directive  1000.04,  (2)  implement  recommendations  that  DOD  Office  of 
Inspector  General  and  GAO  made  to  improve  FVAP’s  oversight  of  and 
direction  to  the  services  as  well  as  to  enhance  its  own  processes,  and  (3) 
evaluate  its  efforts  to  accomplish  its  mission.  Program  evaluation  is  an 
area  where  FVAP  can  continue  to  improve.  Even  though  FVAP  added  new 
performance  measures  and  more  rigor  for  some  evaluative  methods  during 
recent  years,  our  analyses  identified  some  continuing  challenges  that 
FVAP  faces  in  obtaining  reliable,  consistent,  and  credible  evaluative 
information  for  monitoring  and  adjusting  its  efforts  to  address 
requirements  in  the  DOD  directive.  Our  work  showed  that  these 
challenges  are  the  result  of  such  things  as  not  following  survey-related 
guidance  from  the  Office  of  Management  and  Budget,  gathering  data  on 
credible  but  limited  content  issues,  and  not  checking  the  reliability  of  self- 
reported  information.  The  new  FVAP  Director  has  reached  out  to  sources 
in  DOD  and  externally  to  obtain  evaluative  expertise  that  FVAP  has  not 
previously  used,  but  the  outreach  has  been  too  recent  to  determine  its 
impact  on  FVAP’s  evaluative  processes.  However,  delays  in  improving  the 
evaluation  of  FVAP  activities  could  result  in  missed  opportunities  to  both 
improve  the  data  that  FVAP  will  have  following  the  2010  federal  election 
and  make  data-guided  improvements  before  the  2012  election.  Delay  in 
improving  the  evaluation  of  the  VAOs’  training  is  particularly  important 
because  VAOs  occupy  such  a  critical  role  in  helping  FVAP  to  address  the 
directive-specified  requirements.  For  example,  the  absence  of  data 
comparing  the  performance  of  VAOs  who  had  online  versus  in-person 
training  raises  questions  about 

•  the  relative  effectiveness  of  the  two  training  methods, 

•  whether  the  online  training  produces  enough  added  benefit  to  warrant 
its  much  higher  cost  than  remote  training,  and 
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•  whether  newer  technological  alternatives  should  replace  one  or  both  of 
the  current  training  methods. 

If  FVAP  were  to  address  these  areas  that  need  improvements  promptly, 
the  agency  would  also  have  better  data  to  provide  to  Congress  in  response 
to  the  new  requirement  for  an  annual  report  which  was  mandated  in  the 
National  Defense  Authorization  Act  for  Fiscal  Year  2010. 


Recommendations 
Executive  Action 


for 


To  improve  the  effectiveness  of  DOD’s  absentee  voting  assistance  efforts, 
we  recommend  that  the  Secretary  of  Defense  direct  the  Under  Secretary  of 
Defense  for  Personnel  and  Readiness  to  take  the  following  three  actions: 


•  Assess  the  methodologies  currently  in  use  to  gather  evaluative 
information  on  FVAP’s  efforts  to  address  the  requirements  in  DOD 
Directive  1000.04. 

•  Implement,  where  needed,  improved  and  supplemental  program 
evaluation  methodologies  to  address  concerns  such  as  those  that  we 
identified  in  this  report. 

•  Evaluate  current  and  alternative  methods  for  training  VAOs  as  part  of 
federal  elections. 


Agency  Comments 
and  Our  Evaluation 


In  written  comments  on  a  draft  of  this  report,  DOD  concurred  with  two  of 
our  recommendations  and  partially  concurred  with  a  third 
recommendation.  The  department’s  written  comments  are  reprinted  in 
appendix  III. 

DOD  concurred  with  our  recommendation  to  assess  the  methodologies 
currently  in  use  to  gather  evaluative  information  on  FVAP’s  efforts  to 
address  the  requirements  in  DOD  Directive  1000.04.  DOD  stated  that  it  is 
already  developing,  with  the  military  services,  new  metrics  to  evaluate  the 
program’s  effectiveness  in  meeting  the  requirements  of  DOD  Directive 
1000.04,  as  well  as  the  new  responsibilities  identified  in  the  Military  and 
Overseas  Voters  Empowerment  Act.  While  developing  these  metrics  is  a 
step  in  the  right  direction,  it  is  too  soon  to  determine  how  well  and  fully 
the  new  metrics  will  evaluate  the  program’s  effectiveness  to  meet  its 
requirements. 

DOD  partially  concurred  with  our  recommendation  to  implement,  where 
needed,  improved  and  supplemental  program-evaluation  methodologies  to 
address  concerns  such  as  those  that  we  identified  in  this  report.  DOD 
discussed  its  survey  methodologies  only  and  therefore  did  not  indicate 
how  FVAP  might  (1)  address  the  non-survey  concerns  that  we  identified 
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with  its  measures  of  success  and  (2)  supplement  its  current  methodologies 
to  enhance  the  agency’s  program  evaluation  efforts.  For  instance,  FVAP 
did  not  identify  plans  to  conduct  reliability  assessments  or  other 
evaluations  to  promote  data  consistency  and  accuracy.  With  regard  to  its 
surveys,  DOD  stated  that  the  “exceptionally  low”  response  rate  for  the 
overseas  civilian  survey  highlights  the  problems  in  defining  the  actual  size 
and  composition  of  the  overseas  American  population.  Given  the  very  low 
response  rate,  DOD  said  FVAP  is  not  considering  any  of  the  results  of  that 
survey  in  its  post-2008  elections  report  to  Congress.  Additionally,  DOD 
said  that  FVAP  and  the  Defense  Manpower  Data  Center  are,  however, 
attempting  to  refine  and  improve  that  survey  and  will  include  nonresponse 
analysis  for  its  2010  survey.  DOD  further  agreed  with  GAO’s  concern 
regarding  survey  content  and  reported  that  FVAP  has  been  in  close 
discussions  with  the  Defense  Manpower  Data  Center  to  include 
appropriate  questions  in  the  2010  Post-Election  Survey.  DOD  also  noted 
that  it  does  not  believe  the  post-election  surveys  of  active  duty  military 
members  and  the  local  election  officials  suffer  in  the  same  way  as  other 
surveys  do  from  low  response  rates.  DOD  added  that  a  lower  response 
rate  is  not  indicative  of  a  flawed  study,  nor  does  the  lack  of  a  specific 
nonresponse  bias  analysis  indicate  that  the  original  survey  results  are  not 
statistically  valid.  The  department  supported  its  assertions  by  noting  that 
the  Defense  Manpower  Data  Center  has  much  data  on  the  demographics 
of  the  people  that  it  surveys  and  by  stating  that  an  academic  researcher 
found  “no  strong  empirical  relationship  between  response  rates  and 
nonresponse  bias.”  Additional  information  needs  to  be  considered  in 
evaluating  DOD’s  assertions.  First,  GAO  raised  the  issue  of  nonresponse 
bias  analysis  because  2006  OMB-issued  guidance  for  executive  branch 
agencies  states  “Agencies  must  design  the  survey  to  achieve  the  highest 
practical  rates  of  response,  commensurate  with  the  importance  of  survey 
uses,  respondent  burden,  and  data  collection  costs,  to  ensure  that  survey 
results  are  representative  of  the  target  population  so  that  they  can  be  used 
with  confidence  to  inform  decisions.  Nonresponse  bias  analyses  must  be 
conducted  when  unit  or  item  response  rates  or  other  factors  suggest  the 
potential  for  bias  to  occur.”17  Thus,  DOD’s  position  that  nonresponse  bias 
analysis  is  not  warranted  is  contrary  to  an  OMB-prescribed  standard  for 
data  collection.  Second,  if  the  Defense  Manpower  Data  Center  does  not 
use  its  extensive  demographic  data  to  analyze  differences  in  response 
rates  between  groups,  DOD  cannot  draw  conclusions  about  how  well  the 


17  Office  of  Management  and  Budget,  Standards  and  Guidelines  for  Statistical  Surveys 
(September  2006). 
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demographics  of  the  respondents  and  nonrespondents  actually  agreed. 
Third,  DOD’s  summarization  of  the  academic  researcher’s  findings  is 
somewhat  misleading.18  The  researcher  said  that  the  relationship  between 
nonresponse  rate  and  bias  was  “modest.”  More  importantly,  other  points 
in  the  researcher’s  study  noted  that:  “A  synthesis  of  research  studies 
estimating  nonresponse  bias  shows  the  bias  often  present,”  and  that:  “As 
response  rates  decline,  researchers  face  a  growing  obligation  to  mount 
nonresponse  bias  studies  in  order  to  inform  the  evaluation  of  survey 
estimates.”  We  reiterate  our  call  for  FVAP  and  Defense  Manpower  Data 
Center  to  follow  OMB  guidance  to  conduct  the  nonresponse  bias  analysis. 
Continued  use  of  the  evaluative  information  without  quality-assurance 
steps  such  as  assessing  data  reliability  and  potential  nonresponse  bias 
could  limit  DOD’s  ability  to  provide  a  more  complete  picture  of  FVAP’s 
performance,  which  could  impact  the  quality  of  future  reports  to  Congress 
submitted  in  response  to  section  586  of  the  National  Defense 
Authorization  Act  for  Fiscal  Year  2010. 

DOD  concurred  with  our  recommendation  to  evaluate  current  and 
alternative  methods  for  training  VAOs  as  part  of  federal  elections.  DOD 
stated  that  it  is  pursuing  a  different  approach  by  transforming  voting 
assistance  into  a  direct-to-the-voter  assistance  program  of  centralized 
online  tools,  training,  and  assistance  provided  by  FVAP.  For  the  2010 
election  cycle,  FVAP  said  it  is  most  prudent  to  continue  the  traditional 
VAO  program  and  supplement  it  with  direct-to-the-voter  assistance.  With 
regard  to  the  focus  of  our  recommendation  on  the  evaluation  of  VAO 
training,  FVAP  indicated  that  it  is  implementing  post-training  evaluation  of 
its  VAO  workshops  and  will  include  more  detailed  questions  in  its  FVAP’s 
surveys  of  voters  and  VAOs  following  the  2010  election.  Reliance  on 
findings  from  those  post-election  surveys  is  problematic  because  of 
concerns  such  as  response  rates  and  identification  of  the  population. 
Additional  evaluation  methods  could  be  useful,  such  as  testing  samples  of 
participants  on  how  much  knowledge  they  gained  from  the  training. 
Similarly,  structured  gathering  of  participants’  perceptions  (e.g.,  which 
training  topics  need  additional  explanation)  and  analysis  of  that 
information  could  offer  insights  regarding  the  relative  advantages  of 
current  and  alternative  methods  of  VAO  training. 


18  Robert  M.  Groves,  “Nonresponse  Rates  and  NonResponse  Bias  in  Household  Surveys,” 
Public  Opinion  Quarterly,  Vol.  70,  No.  5,  Special  Issue,  pp.  646-675  (2006).  The  cited 
quotes  are  on  pages  646,  657,  and  659. 
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As  we  agreed  with  your  office,  unless  you  publicly  announce  the  contents 
of  this  report  earlier,  we  plan  no  further  distribution  of  it  until  30  days 
from  its  date.  At  that  time,  we  will  send  copies  to  other  interested  parties. 
The  report  will  also  be  available  at  no  charge  on  the  GAO  Web  site  at 
http://www.gao.gov.  Should  you  or  your  staff  have  any  questions  about 
this  report,  please  contact  me  at  (202)  512-8246.  Contact  points  for  our 
Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on  the 
last  page  of  this  report.  Key  contributors  to  this  report  are  listed  in 
appendix  IV. 


Jack  E.  Edwards 

Director,  Defense  Capabilities  and  Management 
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The  Honorable  Robert  Brady 
Chairman 
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Committee  on  House  Administration 
House  of  Representatives 

The  Honorable  Susan  Davis 
Chairwoman 

Subcommittee  on  Military  Personnel 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Mike  Honda 
House  of  Representatives 

The  Honorable  Carolyn  Maloney 
House  of  Representatives 
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To  evaluate  the  extent  to  which  the  Federal  Voting  Assistance  Program 
(FVAP)  has  addressed  its  mission  and  evaluated  the  effectiveness  of  its 
efforts  to  conduct  that  mission,  we  reviewed  and  analyzed  relevant  laws 
(e.g.,  Uniformed  and  Overseas  Citizens  Absentee  Voting  Act — UOCAVA1), 
directives  (e.g.,  Department  of  Defense  (DOD)  Directive  1000.04),  and 
other  guidance  (e.g.,  memorandums).  We  reviewed  UOCAVA  and  DOD 
Directive  1000.04  to  identify  specific  federal  responsibilities  for  absentee 
voting  and  compared  these  responsibilities  with  actions  taken  by  FVAP. 
We  also  reviewed  relevant  reports  prepared  by  GAO,  FVAP,  DOD  Office  of 
Inspector  General,  Election  Assistance  Commission,  Overseas  Vote 
Foundation,  and  Pew  Charitable  Trusts  to  gain  a  thorough  understanding 
of  prior  reviews  conducted  on  FVAP  and  the  challenges  involved  in  the 
absentee  voting  process.  To  assess  the  effectiveness  of  FVAP’s  efforts  to 
conduct  its  mission,  we  reviewed  FVAP’s  data  gathering  procedures  and 
available  measures  obtained  from  FVAP  and  published  reports  (e.g.,  ours 
and  those  of  the  DOD  Office  of  Inspector  General  and  non-governmental 
organizations).  We  then  compared  FVAP’s  procedures  and  measures 
against  criteria  on  performance  measurement,  program  evaluation,  and 
related  issues  that  GAO  and  others  published  in  reports.2  We  additionally 
reviewed  the  methodology  and  questions  used  in  FVAP’s  2008  postelection 
surveys  of  servicemembers,  VAOs,  overseas  citizens,  and  local  election 
officials  and  compared  FVAP-related  information  against  criteria  in 
guidance  from  the  Office  of  Management  and  Budget.3  We  interviewed  key 
survey  program  officials  from  the  Defense  Manpower  Data  Center  to 
obtain  information  and  methodologies  for  FVAP’s  postelection  surveys. 

We  also  interviewed  officials  from  FVAP,  the  Department  of  State,  the 
Election  Assistance  Commission,  Overseas  Vote  Foundation,  and  Pew 
Charitable  Trusts  to  obtain  their  perspectives  on  FVAP’s  efforts. 
Additionally,  we  reviewed  FVAP’s  2010-2011  Voting  Assistance  Guide  and 
the  agency’s  Web  site  to  determine  the  types  of  information  provided  to 
UOCOVA-covered  citizens  considering  or  attempting  to  vote  absentee.  To 
specifically  assess  the  requirement  that  FVAP  train  VAOs,  we  held 
interviews  with  key  officials  and  with  a  nonprobability  sample  of  VAOs 
from  the  services  to  obtain  their  perspective  of  FVAP’s  VAO  training. 
Because  this  was  a  nonprobability  sample  of  VAOs,  the  results  of  these 


1  Pub.  L.  No.  99-410  (1986),  codified  at  42  U.S.C.  §§1973ff  et  seq. 

2  GAO,  Tax  Administration:  IRS  Needs  to  Further  Refine  Its  Tax  Filing  Season 
Performance  Measures,  GAO-03-143  (Washington,  D.C.:  Nov.  22,  2002). 

3  Office  of  Management  and  Budget,  Guidance  on  Agency  Survey  and  Statistical 
Information  Collections  (Washington,  D.C.:  January  2006). 
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interviews  are  not  generalizeable  to  the  population  of  VAOs.  To  select  the 
VAOs,  we  obtained  a  list  of  VAOs  for  the  most  recent  voting  cycle  from  the 
Army,  Navy,  Marine  Corps,  and  Air  Force  and  generated  a  non-probability 
random  sample  from  each  list.  We  observed  in-person  VAO  training 
workshops  and  completed  the  online  training  that  can  also  be  used  for 
VAO  training.  For  both  types  of  training,  we  compared  the  methodological 
and  content-related  information  against  requirements  specified  in  DOD 
Directive  1000.04  and  criteria  specified  in  training  standards  that  we 
published.4 

To  evaluate  the  extent  to  which  FVAP  has  processes  in  place  to  align  its 
budget  priorities  with  its  strategic  plans  and  goals,  we  reviewed  FVAP’s 
budget  development  procedures  and  compared  its  budgets  for  fiscal  years 
2005  through  2009  with  the  goals  described  in  its  strategic  plan  for  2008 
through  2009.  We  began  with  fiscal  year  2005  because  this  is  when  FVAP’s 
budget  reporting  requirement  shifted  from  the  Washington  Headquarters 
Services  to  the  Defense  Human  Resource  Activity.  Additionally,  we 
interviewed  key  FVAP  and  Defense  Human  Resource  Activity  officials  to 
clarify  the  program’s  planning  and  budgeting  processes.  After  gaining  an 
understanding  of  the  budget,  budget  process,  and  contents  of  FVAP’s 
strategic  plan,  we  worked  with  FVAP  staff  to  identify  which  parts  of  the 
budget  were  primarily  used  to  address  each  of  the  goals  in  the  strategic 
plan. 

To  determine  the  extent  to  which  DOD  Office  of  Inspector  General  and 
GAO  recommendations  made  from  2001  through  2009  were  implemented 
by  FVAP,  we  reviewed  DOD  Office  of  Inspector  General  and  GAO  reports 
on  absentee  voting  to  identify  any  recommendations  related  to  this 
program.  5  We  also  interviewed  DOD  Office  of  Inspector  General  officials 
who  conducted  the  evaluations  of  FVAP’s  and  the  Under  Secretary  of 
Defense  for  Personnel  and  Readiness’s  implementation  efforts  and 
reviewed  documentation  on  those  efforts.  Additionally,  we  discussed  our 
findings  with  DOD  Office  of  Inspector  General  officials  to  clarify  the 
recommendations’  status,  actions  taken  by  FVAP  and  the  Under  Secretary, 
and  any  positive  outcomes  from  implementation.  Similarly,  we  reviewed 
our  reports  with  UOCAVA-related  recommendations  as  well  as 


4  GAO,  Human  Capital:  A  Guide  for  Assessing  Strategic  Training  and  Development 
Efforts  in  the  Federal  Government,  GAO-04-546G  (Washington,  D.C.:  Mar.  1,  2004). 

5  The  DOD  Office  of  Inspector  General  reports  were  completed  in  response  to  10  U.S.C. 
§1566. 
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information  later  obtained  to  document  the  efforts  to  implement  our 
recommendations.  In  cases  where  a  recommendation  was  not 
implemented  or  partially  implemented,  we  sought  additional  information 
from  FVAP  officials,  for  example,  confirmation  that  our  finding  was 
correct  and  the  reason  why  the  recommendation  was  not  fully 
implemented.  For  each  of  our  objectives,  we  assessed  the  reliability  of  the 
data  we  analyzed  by  reviewing  existing  documentation  related  to  the  data 
sources  and  interviewing  knowledgeable  agency  officials  about  the  data 
that  we  used.  We  found  the  data  sufficiently  reliable  for  the  purposes  of 
this  report. 

We  conducted  this  performance  audit  from  June  2009  through  May  2010  in 
accordance  with  generally  accepted  government  auditing  standards.  Those 
standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient, 
appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence 
obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives. 
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Department  of  Defense,  Office  of  the  Inspector  General.  2008  Evaluation 
of  the  DOD  Voting  Assistance  Program.  Report  Number  IE-2009-005. 
Arlington,  Virginia:  April  30,  2009. 

Department  of  Defense,  Office  of  the  Inspector  General.  2007  Evaluation 
of  the  Federal  Voting  Assistance  Program  in  the  Department  of  Defense. 
Report  Number  IE-2008-002.  Arlington,  Virginia:  March  31,  2008. 

GAO.  Elections:  Action  Plans  Needed  to  Fully  Address  Challenges  in 
Electronic  Absentee  Voting  Initiatives  for  Military  and  Overseas 
Citizens.  GAO-07-774.  Washington,  D.C.:  June  14,  2007. 

Department  of  Defense,  Office  of  the  Inspector  General,  2006  Evaluation 
of  the  Federal  Voting  Assistance  Program  in  the  Department  of  Defense. 
Report  Number  IE-2007-004.  Arlington,  Virginia:  March  31,  2007. 

Department  of  Defense,  Office  of  the  Inspector  General,  Evaluation  of  the 
Voting  Assistance  Program.  Report  Number  IE-2006-001.  Arlington, 
Virginia:  March  31,  2006. 

Department  of  Defense,  Office  of  the  Inspector  General,  Evaluation  of  the 
Voting  Assistance  Program.  Report  Number  IE-2005-001.  Arlington, 
Virginia:  March  31,  2005. 

Department  of  Defense,  Office  of  the  Inspector  General,  DOD 
Implementation  of  the  Voting  Assistance  Program.  Report  Number  D- 
2004-065.  Arlington,  Virginia:  March  31,  2004. 

Department  of  Defense,  Office  of  the  Inspector  General,  DOD  Compliance 
With  the  Uniformed  and  Overseas  Citizens  Absentee  Voting  Act.  Report 
Number  D-2003-072.  Arlington,  Virginia:  March  31,  2003. 

GAO.  Elections:  Voting  Assistance  to  Military  and  Overseas  Citizens 
Should  Be  Improved.  GAO-01-1026.  Washington,  D.C.:  September  28,  2001. 
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UNDER  SECRETARY  OF  DEFENSE 
4000  DEFENSE  PENTAGON 
WASHINGTON.  D  C.  20301-4000 


PERSONNEL  AND 
READINESS 


KAY  2  4  500 


Mr.  Jack  E.  Edwards 

Director,  Defense  Capabilities  and  Management 
U.S.  Government  Accountability  Office 
441  G  Street.  N.W. 

Washington.  DC  20548 

Dear  Mr.  Edwards: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  draft  report, 
ELECTIONS:  DoD  Can  Strengthen  Evaluation  of  Its  Absentee  Voting  Assistance 
Program,  dated  April  20,  2010  (GAO  Code  35 1362/GAO- 10-476). 

The  Federal  Voting  Assistance  Program  (FVAP)  will  make  strategic  and  operational 
improvements  based  on  the  findings  of  its  2008  post-election  surveys  of  Active  Duty 
Military  and  Local  Election  Officials,  customer  and  stakeholder  feedback,  as  well  as 
input  from  the  GAO  and  DoD  Inspector  General  reports.  GAO  only  had  the  2008-09 
FVAP  Strategic  Plan  available  for  analysis.  FVAP  has  since  initiated  a  strategic  planning 
effort  based  upon  new  ideas  for  voting  assistance  improvements,  the  results  of  the  2008 
post-election  surveys,  and  inputs  from  your  staff  during  this  audit.  This  new  effort  will 
substantially  address  the  issues  raised  in  this  report. 

The  2008  post-election  surveys  show  military  voter  registration  rates  exceed  those  of  the 
national  electorate,  and  that  military  voter  participation  rates,  when  adjusted  for  the 
substantial  age  and  gender  differences  between  military  and  civilian  populations,  also 
exceed  the  national  voter  participation  rates.  Where  military  voter  success  differs  sharply 
from  the  general  electorate  is  in  absentee  ballot  return  rates:  only  63%  of  military 
absentee  ballots  were  returned  for  the  November  2008  general  election,  compared  to  91% 
of  the  general  electorate.  It  appears  the  primary  reason  for  this  failure  is  that  ballots  were 
not  sent  to  military  voters  far  enough  before  the  election  for  military  voters  to  receive, 
complete,  and  return  them  by  State-imposed  absentee  ballot  receipt  deadlines.  This  is 
outside  the  control  of  the  military  Voting  Assistance  Officers,  of  course,  but  significantly 
impacts  overall  military  voter  success  rates.  The  requirement  imposed  upon  States  by  the 
MOVE  Act  to  transmit  ballots  to  military  and  overseas  voters  at  least  45  days  before 
federal  elections,  and  to  transmit  them  electronically,  should  substantially  improve  the 
absentee  ballot  return  rates  for  military  and  overseas  voters. 
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We  believe  the  FVAP  surveys  show  the  FVAP  program  is  successfully  assisting  military 
voters  in  exercising  their  right  to  vote.  We  agree  we  can  do  even  better,  and  are 
committed  to  further  studies  to  monitor  and  improve  what  we  do.  Thank  you  for  your 
cooperation  and  the  opportunity  to  comment  on  this  report. 


Sincerely, 


Clifford  L.  Stanley 


Attachment: 

Comments  to  the  Recommendations 
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GAO  DRAFT  REPORT  -  DATED  APRIL  20,  2010 
GAO  CODE  35 1362/GAO- 10-476 

"ELECTIONS:  DoD  Can  Strengthen  Evaluation  of  Its  Absentee  Voting  Assistance 

Program" 


DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  RECOMMENDATIONS 


RECOMMENDATION  1:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Under  Secretary  of  Defense  for  Personnel  and  Readiness  (USD(P&R))  to  assess  the 
methodologies  currently  in  use  to  gather  evaluative  information  on  Federal  Voting 
Assistance  Program’s  (FVAP’s)  efforts  to  address  the  requirements  in  DoD  Directive 
1000.04. 

DOD  RESPONSE:  Concur.  The  Federal  Voting  Assistance  Program  (FVAP)  is  already 
developing,  with  the  military  Services,  the  new  metrics  to  evaluate  the  program’s 
effectiveness  in  meeting  the  requirements  of  DoD  Directive  1000.04,  to  include  the  new 
responsibilities  put  forward  in  the  Military  and  Overseas  Voters  Empowerment  Act.  As 
noted  in  the  GAO  report,  the  current  draft  of  the  revised  DoD  Instruction  1000.04  will 
address  many  of  the  shortcomings  GAO  identified  in  the  prior  year  program  evaluation 
information.  Further,  the  combined  Unit  Voting  Assistance  Officer  and  dircct-to-the- 
voter  assistance  program  FVAP  has  undertaken  during  the  2010  election  cycle  will 
present  an  opportunity  to  evaluate  both  programs  against  each  other. 

RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Under  Secretary  of  Defense  for  Personnel  and  Readiness  (USD(P&R))  to  implement, 
where  needed,  improved  and  supplemental  program  evaluation  methodologies  to  address 
concerns  such  as  those  that  we  identified  in  this  report. 

DOD  RESPONSE:  Partially  Concur.  The  exceptionally  low  response  rate  for  the 
overseas  civilian  survey  highlights  the  problems  the  Federal  Voting  Assistance  Program 
(FVAP)  and  others  in  both  the  overseas  community  and  federal  government  (Department 
of  State  and  Census  Bureau)  face  in  defining  the  actual  size  and  composition  of  the 
overseas  American  population.  Given  that  very  low  response  rate,  FVAP  is  not 
considering  any  of  the  results  of  that  survey  in  its  post-2008  election  report.  This  is  a 
departure  from  previous  FVAP  practice  which  would  report  such  survey  results, 
regardless  of  the  survey  population  coverage  or  response  rates,  no  matter  how  low. 

To  determine  possible  remedies,  FVAP  and  the  Defense  Manpower  Data  Center  are 
examining  a  number  of  alternative  methods  to  refine  and  improve  the  overseas  civilian 
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survey,  including  revised  survey  methods  and  new  outreach  programs  through  overseas 
citizen  advocacy  groups.  Regardless,  until  the  overall  overseas,  non-federal,  U.S.  citizen 
population  is  better  defined,  it  will  be  impossible  to  fully  evaluate  the  effectiveness  of  the 
voting  assistance  program  provided  to  overseas  Americans;  voter  registration  and  voter 
participation  rates  are  impossible  to  calculate  if  the  denominator  of  the  total  overseas 
American  population  is  unknown. 

In  contrast,  FVAP  does  not  believe  the  active  duty  military  and  the  local  election  official 
post-election  surveys  suffer  in  the  same  way  from  lower  response  rates.  The  Department 
understands  GAO’s  concerns  with  response  rates  and  lack  of  non-response  bias  studies 
during  the  2008  Post-Election  Survey,  but  a  lower  response  rate,  in  and  of  itself,  is  not 
indicative  of  a  flawed  study,  nor  does  the  lack  of  specific  non-response  analysis  indicate 
that  the  original  survey  results  are  not  statistically  valid.  Groves  (2006)  finds  no  strong 
empirical  relationship  between  response  rates  and  nonresponse  bias.1 

FVAP  believes  that  the  active  duty  survey  likely  has  lower  nonresponse  bias  than  surveys 
with  much  higher  response  rates  because  generally  survey  organizations  know  very  little 
about  survey  nonrespondents.  For  instance,  in  telephone  surveys,  for  ‘ring-no-answer’ 
cases  the  surveyor  may  only  know  limited  geographic  data  based  on  the  telephone 
exchange.  For  household  interview  surveys,  the  surveyor  may  only  have  outdated 
knowledge  (usually  Census  data)  of  block  characteristics  (e.g.,  percent  Hispanic).  For  the 
FVAP  survey  of  active  duty  military,  DMDC  has  exceptionally  high  quality  and  timely 
demographic  data  available  for  both  respondents  and  nonrespondents,  including  Service, 
paygrade,  age,  gender,  education,  deployment  history,  and  much  more.  DMDC 
statisticians  use  knowledge  of  nonrespondents’  characteristics  to  create  sophisticated 
statistical  adjustments  designed  specifically  to  reduce  nonresponse  bias,  including  non¬ 
response  and  post-stratification  weighting  adjustments. 

For  the  FVAP  survey  of  local  election  officials  (LEO),  the  55  percent  response  rate  is 
generally  considered  high  for  a  survey  that  does  not  include  high-cost  survey  methods 
like  face-to-  face  interviewing.  For  the  LEO  survey,  we  also  have  considerable 
information  on  nonrespondent  LEOs,  including  state,  jurisdiction  size  in  number  of 
registered  voters,  and  type  of  governmental  unit  (county  versus  local).  These  data  were 
used  both  for  statistical  imputations  for  item-missing  data  and  weighting  adjustments  to 
compensate  for  unit  nonresponse,  and  again  both  these  procedures  reduce  the 
nonresponse  bias  in  survey  estimates. 

For  both  these  surveys,  FVAP  and  DMDC’s  statisticians  consider  estimates  to  be 
representative  of  their  respective  populations  and,  therefore,  the  results  can  be  used  in 
program  evaluation,  policy  decisions,  and  program  planning  and  execution. 


1  Groves,  Robert  M.  “Nonresponse  Rates  and  Nonresponse  Bias  in  Household  Surveys."  Public  Opinion 
Quarterly,  70(5):646-675. 


Page  35 


GAO-10-476  Elections 


Appendix  III:  Comments  from  the  Department 
of  Defense 


Regardless,  the  Department  will  include  non-response  analysis  in  its  2010  survey.  The 
Department  also  agrees  with  GAO’s  specific  concern  regarding  survey  content,  such  as 
local  election  official  views  on  the  effectiveness  of  FVAP's  Legislative  Initiatives 
program,  and  has  been  in  close  discussions  with  the  Defense  Manpower  Data  Center  to 
include  such  questions  in  the  2010  Post-Election  Survey. 


RECOMMENDATION  3:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Under  Secretary  of  Defense  for  Personnel  and  Readiness  (USD(P&R))  to  evaluate 
current  and  alternative  methods  for  training  voting  assistance  officers  (VAOs)  as  part  of 
federal  elections. 

POD  RESPONSE:  Concur.  The  Federal  Voting  Assistance  Program  (FVAP)  also 
agrees  with  the  Department  of  Defense  Inspector  General’s  2005  assessment  of  the 
Voting  Assistance  Officer  program:  “voting  assistance  will  always  be  a  secondary  duty 
[for  military  unit  Voting  Assistance  Officers,  and]  senior  leadership  can  expect 
improvement  only  if  a  radically  different  approach  is  applied.”2  To  that  end,  FVAP  is 
pursuing  a  different  approach  by  transforming  voting  assistance  into  a  direct  to  the  voter 
assistance  program  of  centralized  online  tools,  training,  and  assistance  provided  by 
FVAP. 

However,  for  the  2010  election  cycle,  given  the  apparent  performance  based  success  of 
the  Voting  Assistance  Officer  program/  FVAP  felt  it  most  prudent  to  continue  the 
traditional  VAO  program  and  layer  the  additional  direct-to-the-voter  assistance  on  top  of 
that  unit-  and  installation-level  VAO  assistance,  in  order  to  evaluate  both  against  each 
other.  Additionally,  the  Military  and  Overseas  Voter  Empowerment  Act  requires 
extensive  voting  assistance  at  installation  voting  assistance  offices,  including  additional 
voting  assistance  through  the  National  Voter  Registration  Act  at  every  installation. 
Coupled  with  increased  help  desk  assistance,  the  FVAP  should  begin  to  decrease  the 
demands  on  the  Unit  VAOs  to  only  those  most  in  need  of  assistance. 

Regardless,  based  on  GAO’s  initial  communications,  FVAP  is  already  implementing 
post-training  evaluation  of  VAO  workshops,  and  will  include  more  detailed  contextual 
and  evaluative  questions  in  its  voter  and  Voting  Assistance  Officer  2010  Post-Election 
Surveys. 


2  DoD  1G,  2004  Evaluation,  p.  1 7. 26 

5  For  the  2008  November  general  election,  77%  of  the  military  was  registered  to  vote  as  compared  to  71%  of 
the  national  Citizen  Age  Voting  Population.  When  adjusted  for  the  substantial  age  and  gender  differences  between 
the  military  and  the  general  electorate,  73%  of  the  military  Citizen  Voting  Age  Population  voted  as  compared  to 
63.6%  of  the  national  Citizen  Voting  Age  Population. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  Web  site  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  Web  site, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 

MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Ralph  Dawn,  Managing  Director,  dawnr@gao.gov,  (202)  512-4400 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125 
Washington,  DC  20548 

Public  Affairs 


Chuck  Young,  Managing  Director,  youngcl@gao.gov,  (202)  512-4800 
U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 


Please  Print  on  Recycled  Paper 


